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EXECUTIVE  SUMMARY 

The  body  of  this  report  details  how  152  general 
aviation  pilots  were  assessed  to  determine  the  weights 
(worth  functions)  they  assigned  to  the  variables  of 
terrain,  ceiling,  visibility  and  precipitation.  The  ob¬ 
tained,  analysis  results  were  sufficiently  clear  to  allow 
the  identification  (at  least  tentatively)  of  distinctly 
different  worth  functions.  Specifically,  four  different 
worth  functions  were  identified  and  characterized  for 
non-mountainous  and  mountainous  terrain  types. 
Two  different  worth  functions  were  identified  and 
characterized  for  over-water  terrain.  Use  of  weather 
information  was  consistent  with  expert  assessment  of 
the  safety  risk  associated  with  VFR  flight  under  vary¬ 
ing  ceiling,  visibility,  and  precipitation  conditions 
over  non-mountainous,  mountainous,  and  water  ter¬ 
rain.  However,  while  use  is  consistent,  pilots  vary  in 
their  expressed  degree  of  comfort  in  flying  over  the 
different  terrain  under  different  weather  conditions. 
These  differences  in  comfort  level  may  occur,  in  part 
at  least,  because  of: 

•Differences  of  understanding  of  the  risks  associ¬ 
ated  with  flying  under  differing  above-minimum 
levels  of  visibility,  ceiling,  and  precipitation  and 
how  terrain  type  impacts  use  of  such  information. 

•Differences  in  their  self-assessment  and  percep¬ 
tion  of  their  ability  and/or  skill  in  flying  under  the 
varying  conditions. 

In  summary,  this  study  found: 

•  It  is  possible  to  capture  reliably  pilots*  “comfort” 
policies  in  terms  of  emphasis  upon  terrain,  ceiling, 
visibility,  and  precipitation  using  a  linear  model¬ 
ing  approach. 

•Pilots  use  compensatory  judgment  rules  with  so¬ 
phisticated  interactions. 

•Pilots  generally  agree  on  how  to  prioritize  the 
safety  implications  of  terrain/weather  conditions. 


•Pilots’  policies  tend  to  fall  cleanly  into  stable 
policy  groups  although  the  individual  pilots  who 
hold  those  views  switch  groups  according  to  ter¬ 
rain. 

•  Pilots’  policies  vary  based  upon: 

1.  General  comfort  level 

2.  A  mixture  of  age  and  general  experience 

3.  Number  of  hours  flown  in  last  90  days 

4.  Reasons  for  flying  (employed  as  pilot  versus 
pleasure  flying) 

IMPLICATIONS 

•  Pilots’  use  of  weather  information  is  consistent 
with  expert  assessment  of  the  safety  risk  associated 
with  VFR  flight  under  varying  ceiling,  visibility, 
and  precipitation  conditions  over  non-mountain¬ 
ous,  mountainous,  and  water  terrain. 

•While  use  is  consistent,  pilots  vary  in  their  ex¬ 
pressed  degree  of  comfort  in  flying  over  the  differ¬ 
ent  terrain  under  different  weather  conditions. 

•Training  or  other  interventions  addressing  risk 
assessment  and  self-perception  would  appear  to  be 
appropriate.  Present  use  of  the  Five  Hazardous 
Thought  Patterns  is  one  method  of  addressing  self 
perception.  Training  directed  at  improving  pilot 
understanding  of  the  effects  of  terrain-type  on 
interaction  of  meteorological  conditions  is  an¬ 
other  potential  source.  Increased  emphasis  on  risk 
assessment  and  self  perception  training  and  exer¬ 
cises  in  initial  training  and  in  subsequent  seminars 
(FAA-sponsored  and  otherwise)  would  seem  to  be 
warranted. 

•  Although  the  compensatory  decison  model  is  used 
by  expert  and  by  many  other  pilots  as  well,  its  use 
in  all  situations  may  not  be  advisable.  Some  care 
and  consideration  must  be  observed  in  training 
inexperienced  pilots  to  use  the  decison  models  of 
expert  pilots,  as  their  injudicious  use  may  be  a 
threat  to  aviation  safety. 
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The  Use  of  Weather  Information  in  Aeronautical  Decision-Making 


INTRODUCTION 

A  National  Transportation  Safety  Board  report 
(NTSB,  1989)  reviewed  361  general  aviation  acci¬ 
dents  that  occurred  between  1983  and  1987  in  which 
Visual  Flight  Rules  (VFR)  into  instrument  meteoro¬ 
logical  conditions  (IMC)  was  listed  as  a  probable 
cause  or  a  related  factor.  (Most  accidents  have  mul¬ 
tiple  causes/factors  assigned  as  probable.) 

Ninety-seven  percent  of  the  probable  causes  were 
attributed  to  the  flight  crew,  and  42%  of  these  cited 
the  manner  in  which  weather  information  was  ob¬ 
tained  (or  not  obtained),  assimilated,  and  used.  There 
were  276  fatal  accidents,  resulting  in  583  fatalities. 

An  earlier  review  of  aircraft  accident  data  (Jensen, 
1982)  reported  similar  data  and  concluded  that  80  to 
85%  of  aircraft  accidents  can  be  assigned  broadly  to 
pilot  error.  The  review  also  indicated  that  51.6%  of 
the  fatal  and  35 . 1  %  of  the  non-fatal  accidents  resulted 
from  faulty  decision-making.  Because  of  the  promi¬ 
nence  of  pilot  error  and  flight  from  VFR  into  IMC  as 
causes  of  general  aviation  accidents,  the  Federal  Avia¬ 
tion  Administration  (FAA)  sponsors  training  inter¬ 
ventions  on  pilot  decision-making  which  address  both 
factors.  The  interventions  focus  on  both  cognitive 
and  affective  (or  motivational)  components  of  pilot 
decision-making  (Brecke,  1981;  Jensen,  1982;  Jensen, 
Adrion,  &  Lawton,  1987).  These  two  components 
differentiate  among  processes  pilots  employ  in  using 
and  evaluating  information  available  to  them  and 
making  choices  about  actions  to  take.  The  cognitive 
component  deals  with  the  processes  by  which  pilots 
establish  and  evaluate  the  alternatives  in  a  decision¬ 
making  situation.  Jensen  (1982)  proposed  that  this 
component  relates  to  pilot  ability  to  search  for  and 
establish  the  relevance  of  all  available  information 
regarding  a  situation,  to  specify  alternative  courses  of 
actions,  and  to  determine  expected  outcomes  from 
each  alternative.  This  component  invokes  the  intel¬ 
lectual,  discriminative  abilities  which  depend  upon 
human  capabilities  to  sense,  store,  retrieve,  and  inte¬ 
grate  information. 


From  a  cognitive  perspective,  according  to  Jensen 
(1982),  pilots  should  discriminate  among  situational 
dimensions.  He  asserts  that  for  discriminative  judg¬ 
ment,  these  important  questions  must  be  addressed: 
1)  Do  pilots  consider  all  situational  dimensions  and 
do  they  assign  proper  weights  to  the  data  elements  for 
each  of  these  dimensions?  2)  Do  pilots  efficiently 
integrate  the  relevant  information  from  the  data  ele¬ 
ments? 

Cognitive  training  provides  information  designed 
to  increase  pilot  awareness  of  flight  safety  under 
adverse  conditions  and  to  assist  pilots  in  assigning  and 
integrating  proper  weights  to  the  data  elements.  Train¬ 
ing  consists  of  exercises  demonstrating  the  risks  asso¬ 
ciated  with  flying  activities  and  the  underlying 
behavioral  causes  of  typical  accidents.  The  effects  of 
stress  on  decision-making  and  the  management  of 
stress  are  also  important  training  elements.  Informa¬ 
tional  materials  include  data  about  such  factors  as 
causes  of  accidents  by  phase  of  operation  and  pilot 
experience,  the  aeronautical  decision-making  pro¬ 
cesses,  and  weather  forecast  accuracy  and  use.^  Infor¬ 
mation  about  weights  pilot  attribute  to  flight  data 
elements,  however,  is  not  available  for  use  in  training 
or  other  interventions. 

According  to  Jensen  et  al.  (1987),  use  of  aeronau¬ 
tical  decision-making  interventions  for  instrument 
pilots  produces  substantial  reductions  in  pilot  error 
rates.  Reduction  in  decisional  mistakes  attributed  to 
the  training  ranges  from  10  to  50%.  Similar  training 
exists  for  student,  private,  commercial,  and  multi¬ 
crew  aircraft  pilots. 

A  very  limited  amount  of  research  on  the  cognitive 
processes  of  pilot  decision-making  has  been  reported. 
This  research,  however,  has  two  important  findings. 
First,  the  feasibility  of  investigating  the  intellectual 
processes  in  decision-making  through  assessment  of 
pilot  worth  or  values  pilots  attribute  to  flight  data 
elements  has  been  demonstrated.  Second,  pilots  ap¬ 
pear  to  weight  flight  data  elements  differentially. 
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because  of  such  factors  as  experience  and  training. 
Flathers,  Giffin,  &  Rockwell  (1982)  presented  pilots 
with  a  diversion-decision  scenario  requiring  an  in¬ 
flight  decision  to  change  destinations  during  a  cross¬ 
country  flight,  because  of  equipment  malfunction. 
Subjects  were  presented  alternative  landing  sites  vary¬ 
ing  according  to  weather  conditions,  air  traffic  con¬ 
trol  facilities,  approach,  and  flight  time  to  reach  the 
sites.  Two  levels  of  each  variable  were  represented. 
Subjects  rank-ordered  the  sites  according  to  their 
preference  of  diversion  landing  site.  Conjoint  analysis 
was  employed  to  derive  worth  functions  for  each  of 
the  four  variables.  Comparisons  of  the  worth  func¬ 
tions  among  pilots  with  differing  experience  and 
training  (e.g.,  flight  hours,  source  of  certification) 
revealed  that  pilots  differentially  weighted  the  vari¬ 
ables.  Interactions  among  the  four  variables  were  not 
found.  Flathers  etai  also  referenced  a  study  by  Curry 
(1976)  that  reported  similar  results  from  a  study  of 
approach-to-landing  decision  making.  These  results 
suggest  that  pilot  error  in  decision  making,  one  of  the 
two  most  frequently  cited  probable  causes  of  general 
aviation  accidents,  may,  in  part,  occur  because  pilots 
do  not  assign  proper  weights  to  the  flight  data  ele¬ 
ments  that  are  available  to  them. 

The  present  study  sought  to  extend  this  area  of 
research  through  an  investigation  of  the  values  or 
worth  functions  pilots  attribute  to  weather  and  terrain 
variables  in  making  decisions.  The  specific  objectives 
were  as  follows: 

•Identify  individual  weights  or  worth  functions 
pilots  attribute  to  ceiling,  visibility,  precipitation, 
and  terrain  flight  data  elements  in  three  represen¬ 
tative  cross-country  flights  in  a  small  aircraft. 

•Assess  interaction  effects  on  assignment  of  weights 
that  may  affect  the  worth  functions  of  the  data 
elements. 

•Assess  whether  pilots  differentially  weight  these 
data  elements,  because  of  such  personal  factors  as 
age,  flying  hours,  and  source  of  certification. 


METHOD 

General  Considerations 

The  research  reported  here  employed  a  scenario- 
based  judgment  task  designed  to  elicit  pilot  worth 
functions  for  visibility,  ceiling,  precipitation,  and 
terrain  variables.  Behavioral  decision  research  shows 
that  task  design  is  crucial  for  making  consistent  judg¬ 
ments.  Depending  upon  the  characteristics  of  a  judg¬ 
ment  task,  individuals  sometimes  use  a  compensatory 
strategy  and  other  times  employ  noncompensatory 
strategies  in  making  decisions.  A  compensatory  strat¬ 
egy  is  one  which  processes  all  relevant  information 
and  trades  off  the  good  and  bad  aspects  of  each 
alternative.  Noncompensatory  strategies  avoid  trade¬ 
offs  among  values  by  reducing  information  process¬ 
ing  demands  by  ignoring  potentially  relevant  problem 
information,  such  as  selecting  a  single  attribute  and 
sorting  on  that  alone. 

A  major  concern  was  to  design  a  task  that  would 
avoid  use  of  noncompensatory  cognitive  strategies  by 
the  subjects  when  they  were  making  their  judgments. 
Subjects  who  employ  cognitive  decision  strategies 
that  are  artifacts  of  the  problem  task,  such  as  restruc¬ 
turing,  application  of  simple  heuristics  (representa¬ 
tiveness,  availability,  anchoring)  and  other  more 
complex  noncompensatory  processes  (e.g.,  elimina- 
tion-by  aspects,  lexicographic)  usually  are  trying  to 
reduce  cognitive  effort.  As  a  result,  subjects  frequently 
neglect  information  relevant  to  the  decision  problem 
(Payne,  Bettman,  &  Johnson,  1988;  Payne,  Bettman, 
&  Johnson,  1992). 

The  task  was  intended  to  facilitate  consistent,  com¬ 
pensatory  decision  strategy  utilization  by  the  respon¬ 
dents  to  make,  using  Jensen’s  (1982)  terminology, 
rational  judgments.  Rational  judgment  implies  that 
pilots  should  use  weighted-additive  strategy  processes. 
In  the  weighted-additive  process  pilots  should  be 
expected  to  consider  the  values  of  each  element  or 
attribute;  weight  the  values  of  each  element  by  its 
importance  for  a  safe  flight;  and  sum  the  weighted 
values  as  a  basis  for  overall  evaluation  of  each  flight 
scenario.  In  practice,  however,  some  pilots  may  be 
expected  to  have  developed  mixed  cognitive  strategies 
(from  their  experience  or  training)  for  considering 
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data  elements.  While  some  may  have  weighted- 
additive  strategies  which  they  consistently  apply  to 
judgments,  other  pilots  may  have  developed  rules 
which  they  apply  to  some  of  their  judgments  that  give 
the  appearance  of  some  non-compensatory  strategy  or 
heuristic,  such  as  elimination-by-aspects.  For  example, 
some  pilots  may  have  developed  a  simple  heuristic 
that  they  will  not  make  any  flight  over  water  regardless 
of  weather  conditions;  however,  weather  conditions 
may  have  important  values  when  flying  over  other 
kind  of  terrain.  The  possibility  of  multiple  strategies 
was  addressed  in  the  analytic  methodology  (Appendix 
F)  employed  by  providing  for  distinguishing  between 
“real”  multiple  policies  and  those  that  may  have  been 
artifacts  of  the  task  itself. 

Design  Implications  of  Behavioral  Decision 
Research 

Design  of  the  decision  task  was  based  on  the  results 
of  behavioral  decision  research  which  indicates  that 
differences  in  decision  strategies  within  a  problem 
task  occur,  according  to  Payne  (1982),  because  “...in¬ 
formation  processing  in  decision  making,  as  in  other 
areas  of  cognition,  is  highly  contingent  upon  the 
demands  of  the  task”  (p.  382).  Different  decision 
strategies  can  be  thought  of  as  different  rules  for 
conducting  the  search  among  the  attributes  and  alter¬ 
natives  (Payne,  Bettman,  &  Johnson,  1988).  To 
reduce  the  likelihood  of  subjects  in  this  study  using 
strategy  variations  which  are  induced  by  the  nature  of 
task  itself  (versus  using  their  own  decision  sets), 
careful  attention  was  given  to  response  mode  and  task 
and  context  characteristics  (Einhorn  &  Hogarth  1981; 
Payne,  Bettman,  &  Johnson,  1988;  Payne,  Bettman, 
&  Johnson,  1992). 

Judgment  versus  Choice  Response  Mode.  Judgment 
and  choice  tasks  require  different  cognitive  processes. 
In  a  judgment  task,  subjects  are  provided  successive 
presentations  of  alternatives  to  each  of  which  a  subject 
assigns  a  value  reflecting  its  psychological  worth. 
Sometimes  the  values  are  in  terms  of  a  rating  scale; 
sometimes  the  psychological  worth  might  be  expressed 
in  terms  of  the  amount  of  money  the  subject  would 
pay  for  the  alternative.  In  a  choice  task,  subjects  are 


presented  with  two  or  more  alternatives  and  asked  to 
select  the  alternative  most  preferred  or  the  order  in 
which  the  alternatives  are  preferred. 

Some  of  the  clearest  examples  of  decision  reversals 
have  been  noted  between  judgment  and  choice  tasks. 
Subjects  reversed  their  decision  in  tasks  containing 
identical  information  as  a  function  of  whether  they 
were  required  to  make  judgments  among  the  dimen¬ 
sions  or  make  choices  among  the  dimensions  (Payne, 
1982).  Because  of  these  findings,  a  judgment  task  was 
used  in  this  research.  In  judgment  tasks,  subjects  seem 
to  select  an  attribute  of  an  alternative  as  an  anchor  and 
then  adjust  that  anchor  based  on  information  associ¬ 
ated  with  other  attributes.  Such  processing  is  more 
holistic.  The  choice  mode  seems  to  involve  primarily 
dimensional  processing  by  which  dimensions  of  one 
alternative  are  compared  with  parallel  dimensions  of 
other  alternatives. 

Task  and  Context  Task  Factors.  Task  and  context 
factors  cause  different  aspects  of  the  problem  to  be 
salient  and  evoke  different  processes  for  combining 
information  (Payne,  Bettman,  &  Johnson,  1992,  p. 
90;  Payne,  1982;  Einhorn  &  Hogarth,  1981).  Task 
factors  are  general  characteristics  of  a  decision  problem 
which  do  not  depend  upon  the  particular  values  of  the 
attributes  or  alternatives,  such  as  the  number  of 
alternatives,  number  of  attributes,  information  display, 
and  time  limits.  Context  factors,  such  as  similarity  of 
alternatives,  are  associated  with  the  particular  values 
of  the  alternatives.  The  following  task  factors  guided 
development  of  the  experimental  task: 

•In  general,  cognitive  effort  in  responding  to  a 
decision  task  seems  to  increase  as  the  number  of 
alternatives  presented  for  each  choice  problem 
increases.  Subjects  tend  to  employ  compensatory 
types  of  decision  strategy  when  choices  are  made 
between  only  two  alternatives.  As  alternatives  are 
added,  subjects  tend  to  use  choice  strategies,  such 
as  the  elimination-by-aspects  rule.  In  the  task 
employed  in  this  research,  judgments  among  the 
weather-terrain  scenarios  was  not  required.  Sub¬ 
jects  considered  each  weather-terrain  scenario  in¬ 
dependently  and  judged  their  comfort  level  about 
flight  under  the  conditions  described. 
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•  In  general,  increases  in  the  number  of  attributes  or 
dimensions  (e.g.,  the  amount  of  information)  of 
alternatives  increases  the  variability  of  strategy  and 
decreases  quality  of  judgments.  Subjects  are  likely 
to  use  some  decision  rule  (e.g.,  elimination  by 
aspects)  when  faced  with  complex  choices.  The 
design  of  the  problem  task  in  this  research  limited 
each  scenario  to  four  attributes  or  dimensions. 

•Time  pressure  seems  to  affect  task  complexity. 
Measures  of  information  search  showed  that  sub¬ 
jects  tend  to  spend  more  time  observing  negative 
information  under  time  pressure.  Although  no 
time  limit  was  imposed,  most  subjects  in  the  study 
reported  here  completed  the  task  in  less  than  50 
minutes  (including  time  for  administrative  in¬ 
structions). 

•Information  display  also  may  affect  judgment- 
choice  behavior.  Use  of  attribute  information  was 
reported  as  increased  when  this  information  was 
displayed  in  an  organized  manner  and  that  infor¬ 
mation  acquisition  (about  attributes)  tends  to 
proceed  in  a  way  consistent  with  the  display  for¬ 
mat.  Studies  have  demonstrated  that  displays  de¬ 
signed  to  encourage  alternative-based  processing 
tended  to  produce  more  alternative-based  infor¬ 
mation  processing.  Also,  completeness  of  the  in¬ 
formation  displayed  is  important.  Subjects  may 
tend  to  discount  alternatives  when  there  is  missing 
information  about  one  or  more  attributes.  The 
judgment  task  presented  dimension  information 
consistently  across  all  scenarios.  Each  scenario 
contained  the  same  dimensions.  Although  values 
varied,  dimensions  appeared  in  the  same  order  and 
format.  Other  data  elements  than  those  repre¬ 
sented  in  the  dimensions  which  could  be  impor¬ 
tant  in  making  judgments  about  the  scenarios 
(e.g.,  equipment  status,  time  of  day,  pilot  physical 
and  psychological  condition)  were  held  constant 
across  all  scenarios  for  two  reasons.  First,  the 
appearance  of  missing  information  was  avoided. 
Second,  variations  in  these  elements  would  have 
introduced  uncontrolled  variables  into  the  design. 


•  The  only  context  variable  that  could  have  implica¬ 
tions  for  the  proposed  research  relates  to  the  simi¬ 
larity  of  dimensions  or  attributes.  It  seems  clear  that 
in  choice  tasks  where  choices  among  alternatives  is 
the  task,  similarity  of  attributes  among  alternatives 
increases  the  cognitive  effort  required  to  make  choices. 
The  effort  required  to  make  comparisons  among 
alternatives  essential  to  choice-making  is  increased, 
thus  inducing  some  non-compensatory  strategy,  such 
as  elimination-by-aspects  (Payne,  1982).  Since 
judgment  of  each  scenario  was  made  indepen¬ 
dently,  the  effects  of  similarity  should  have  been 
eliminated  or  minimized. 

Data  Collection  Instrument  Development 

Data  collection  instruments  consisted  of  a)  a  test 
booklet  (used  to  obtain  pilot  background  and  experi¬ 
ence  information),  b)  aset  of  laminated  colored  maps 
(one  for  each  of  three  terrain  types:  water,  non- 
mountainous,  and  mountainous),  and  c)  three  sets  of 
en  route  weather  scenarios  (each  set  corresponding  to 
a  terrain  type).  Appendix  A  is  a  set  of  these  data 
collection  instruments. 

Test  Booklets.  Booklets  consisted  of  three  separate 
parts.  Part  one  contained  instructions  for  assigning 
“comfort”  ratings  to  the  weather  scenarios.  Part  two 
elicited  demographic  information  such  as  certification 
source,  age,  total  flying  hours,  total  hours  flown  in  the 
last  90  days.  This  demographic  information  paralleled 
that  cited  by  the  National  T ransportation  Safety  Board 
(NTSB,  1989)  as  being  related  to  aircraft  accidents 
the  NTSB  studied  from  1983  to  1987.  Part  three  of 
the  booklet  was  designed  to  obtain  the  pilot-subjecfs 
degree  of  experience  flying  over  each  of  the  three  types 
of  terrain,  and  their  experience  flying  into  various 
levels  of  precipitation  (light  rain,  moderate  rain,  heavy 
snow),  visibility  (1  NM,  4  NM,  8  NM),  and  ceiling 
(800  ft,  1800  ft,  4000  ft).  A  5'point  scale  (0=1  have 
never  flown  into  these  or  highly  similar  conditions; 
4=1  have  flown  into  these  conditions  numerous  times 
(11+))  was  used  for  these  ratings. 

Terrain  Maps/Flight  Routes.  The  terrain  maps  selected 
were  for  the  Great  Lakes  (over  water).  North  Texas  (non- 
mountainous)  and  New  Mexico  (mountainous)  areas 
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and  were  clearly  identifiable  with  respect  to  these 
terrain  types.  In  the  selection  of  the  over-water  terrain 
map,  southern  coastal  areas  were  removed  from 
consideration  because  of  limitations  imposed  on  the 
development  of  realistic  weather  scenarios  (e.g.,  snow 
would  not  be  a  condition  normally  encountered  over 
southern  coastal  areas).  It  was  felt  that  local  knowledge 
of  terrain,  if  possessed  by  some  subjects  and  not 
others,  would  act  as  an  uncontrolled  independent 
variable.  The  importance  of  minimizing  this  local 
knowledge  was  emphasized  by  the  Shanteau  and  Nagy 
(1979)  research  which  indicated  that  subjects’  previous 
knowledge  about  a  task  to  be  performed  influenced 
their  choices  in  task  performance.  Since  pilots  were  to 
be  drawn  from  the  San  Antonio,  Texas  area,  the  route 
map  selected  (Great  Lakes,  North  Texas,  and  New 
Mexico)  minimized  the  potential  impact  of  local 
knowledge. 

Routes  to  be  flown  (City  County  to  Delta  County, 
Michigan  for  over  water,  Hereford  to  Dalhart,  Texas 
for  non-mountainous,  and  Las  Vegas  to  San  Juan 
Pueblo,  New  Mexico  for  mountainous)  were  laid  out 
to  minimize  impact  of  restricted,  special-use  airspace, 
and  low-level  military  training  routes.  These  flight 
routes  were  clearly  marked  on  the  corresponding  color 
terrain  map.  In  addition  to  flight  routes,  maps  were 
also  annotated  with  local  and  destination  weather  and 
a  description  of  the  type  and  condition  of  the  aircraft 
being  flown  (e.g.,  Cessna  172  without  transponder 
and  navigation  radio  out). 

Scenario  Development.  En  route  weather  scenario 
content  was  developed  from  the  NTSB  (1989)  study 
of  weather-related  general  aviation  accidents  from 
1983  to  1987.  In  addition  to  a  probable  cause  ofVFR 
flight  into  IMC  conditions,  which  was  the  criterion 
used  by  the  NTSB  to  select  accidents  for  analysis, 
89%  ofthese  accidents  were  attributed  to  the  following 
four  factors: 

Terrain  conditions  225  (18.6%) 

Low  ceiling  216  (17.9%) 

Poor  visibility  448  (37.2%) 

Precipitation  180  (15.3%) 


As  a  result  of  these  findings,  scenarios  were  gener¬ 
ated  (using  a  multi-stage  process)  around  variations  in 
terrain,  ceiling,  visibility,  and  precipitation.  Specifi¬ 
cally,  81  scenarios  were  produced  to  accommodate 
three  levels  of  each  of  these  four  independent  at¬ 
tributes.  Each  scenario  was  printed  on  a  5  by  4  inch 
card,  color  coded  with  respect  to  terrain  type,  and 
described  a  combination  of  en  route  weather  condi¬ 
tions  (ceiling,  visibility,  and  precipitation)  encoun¬ 
tered  during  the  cruise  phase  of  flight  over  each  of  the 
three  terrain  types. 

The  first  stage  of  scenario  development  focused  on 
the  identification  and  validation  of  the  three  levels 
(low,  medium,  and  high)  of  ceiling,  visibility,  and 
precipitation  variables  to  be  used  in  scenario  develop¬ 
ment.  Specifically,  an  Expert  Pilot  Safety  Ratings 
Form  (Appendix  C)  was  developed  and  administered 
to  22  pilot-subjects.  This  group  of  pilots  had  an 
average  of  3,700  hours  of  total  flight  time  and  had  an 
average  age  of  42  years.  Of  the  22  pilots,  16  were 
certified  flight  instructors  and  1 1  were  employed  as 
pilots.  These  22  pilots  used  a  O-to-100  point  scale 
(0=Absolutely  unsafe/1 00=totally  safe)  to  rate  the 
relative  safety  of  the  three  routes  (over  water,  non- 
mountainous,  and  mountainous)  and  of  levels  of 
ceiling,  visibility,  and  precipitation  with  respect  to 
VFR  flying  by  a  pilot  with  500  hours  of  flying  expe¬ 
rience. 

These  22  pilot-subjects  first  provided  safety  ratings 
for  8  levels  of  precipitation  (ranging  from  no  precipi¬ 
tation  to  freezing  rain),  12  levels  of  visibility  (ranging 
from  Vi  NM  to  more  than  8  NM),  and  16  levels  of 
ceiling  (ranging  from  600  feet  to  over  5000  feet). 
Interrater  agreements  for  the  obtained  ratings  (stan¬ 
dardized  to  a  mean  of  5.0  and  a  standard  deviation  of 
1.0)  were  computed  using  the  GRPREL  function  of 
the  Comprehensive  Occupational  Data  Analysis  Pro¬ 
grams  (CODAP;  Christal,  1974;  Staley  and 
Weissmuller,  1981).  (A  brief  description  of  CODAP 
and  its  components  is  provided  in  Appendix  F.)  The 
interrater  agreements,  shown  in  T able  1 ,  were  extraor¬ 
dinarily  high.  The  interrater  agreement  (R^^)  indi¬ 
cates  the  degree  to  which  pilots  within  the  policy 
group  agreed  on  how  to  sort  and  rate  the  81  weather 
scenarios  with  respect  to  comfort  level.  Values  of  0.20 
and  above  are  considered  “good  agreement”  (Staley 
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Table  1.  Reliability  of  the  Expert  Pilot 
Safety  Ratings. 


Table  2.  Standardized  High,  Medium  and 
Low  Levels  of  Safety  Variables. 


Variable  Levels 

N 

Raters 

Rii 

Rkk 

Routes 

22 

.67 

.97 

Precipitation  Levels 

22 

.88 

.99 

Visibility  Levels 

22 

.92 

.99 

Ceiling  Levels 

22 

.92 

.99 

All  Variable  Levels 

22 

.82 

.99 

and  W eissmuller,  1981).  The  policy  stability  measure 
indicates  the  expected  correlation  of  this  policy 
with  the  policy,  which  would  be  found  in  a  re-survey 
of  “k”  subjects.  Over  all  variables  and  levels,  was 
.82  and  Rj^j^  was  .99. 

To  identify  values  associated  with  the  three  levels 
of  precipitation,  visibility,  and  ceiling,  means  and 
standard  deviations  were  computed,  standardized  to  a 
mean  of  5.0  and  a  standard  deviation  of  1.0  for  each 
of  the  three  attributes,  and  then  plotted.  Sets  of 
“High,”  “Medium,”  and  “Low”  values  were  then 
identified  (by  inspection)  for  each  level.  The  results, 
shown  in  Table  2,  represented  a  reasonable  spread 
across  the  range  of  values.  Appendix  D  contains  the 
associated  plots  used. 

Three  sets  of  27  weather  scenarios  were  then  gener¬ 
ated  (one  set  for  each  terrain  type).  Specifically,  sce¬ 
nario  sets  were  initially  generated  such  that  each 
scenario  within  a  set  was  unique  with  respect  to 
precipitation,  visibility,  and  ceiling  level  combina¬ 
tion  (e.g.,  High/High/High,  High/High/Medium, 
etc.).  Within  level,  however,  selection  of  the  specific 
value  was  randomized  i.e.,  selection  of  5000  ft,  4000 
ft,  or  3500  ft  within  the  high  level  of  the  ceiling 
variable  was  randomized.  Each  set  of  scenarios  was 
then  randomly  numbered  (600-to-626for  non-moun- 
tainous,  700-to-726  for  mountainous,  and  800-to- 
826  for  water). 

Prior  to  finalizing  the  three  sets  of  scenarios,  each 
set  was  evaluated  with  respect  to  plausibility.  Subject- 
matter  expert  judgment  was  used  for  this  purpose. 
Specifically,  12  weather  experts  (meteorologists  and 


Variable 

Benchmark 

value/Levet 

Precipitation;  ' 

none 

6.745/High 

light  rain 

6.000/High 

light  snow 

5.514/Medium 

moderate  rain 

5.205/Medium 

moderate  snow 

4.687/Medium 

heavy  rain 

4.287/Low 

heavy  snow 

3.975/Low 

freezing  rain 

3.589/Low 

Visibility;  : 

more  than  8  NM 

6.598/High 

8NM 

6.410/High 

7NM 

6.165/High 

5NM 

5.446/Medium 

4NM 

4.939/Medium 

3NM 

4.639/Medium 

1  1/2  NM 

3.960/Low 

1  NM 

3.896/Low 

1/2  NM 

3.822/Low 

CeHing:  ,,  .  i  .: 

5000  ft 

6.419/High 

4000  ft 

6.244/High 

3500  ft 

6.081/High 

2000  ft 

5.393/Medium 

1800  ft 

5.010/Medium 

1600  ft 

4.702/Medium 

900  ft 

3.928/Low 

800  ft 

3.818/Low 

600  ft 

3.664/Low 

weather  observers  from  the  National  Weather  Service 
Forecast  Office  in  San  Antonio,  Texas,  and  USAF 
Weather  Detachments  at  Reese  AFB,  Texas,  and 
Randolph  AFB,  Texas)  were  asked  to  rate  the  degree 
to  which  scenario  weather  combinations  (precipita¬ 
tion,  visibility,  and  ceiling  values)  could  reasonably 
be  expected  to  occur  together  (e.g,,  heavy  snow,  5NM 
visibility,  and  a  ceiling  of  3500  feet  would  not  be 
expected  to  occur  simultaneously).  Subjects  rated 
each  scenario  on  a  O-to-100  scale  (0=Totally  improb¬ 
able/ 100=  Very  probable).  Obtained  interrater  agree¬ 
ment  was  high  —  R^^  was  .63  and  R^^j^  was  .95. 
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Analysis  of  the  resulting  mean  ratings  led  to  the 
replacement  of  12  of  81  draft  scenarios.  Scenarios 
with  mean  ratings  (standardized  to  a  mean  of  5.0  and 
a  standard  deviation  of  1.0)  more  than  1.5  standard 
deviations  below  the  mean  rating  were  considered 
improbable  and  replaced.  Four  scenarios  from  the 
non-mountainous  set,  three  from  the  mountainous 
set,  and  five  from  the  over  water  set  were  replaced 
because  of  low  mean  ratings.  The  replacement  sce¬ 
nario  within  a  given  set  was  drawn  from  one  of  the 
other  two  sets.  The  scenario  with  the  lowest  accept¬ 
able  mean  value  (of  those  scenarios  not  already  used  as 
replacements)  was  always  selected  as  the  replacement 
scenario.  This  process  ensured  that  (to  the  extent 
possible)  an  equivalent  level  weather  combination  was 
substituted  for  the  unrealistic  weather  combination. 

Pilot-Subjects 

A  total  of  152  pilots  were  tested  from  January  to 
March  1994.  Complete  data  were  not  available  for 
two  pilots  who  were  subsequently  dropped  from  the 
study.  The  remaining  pilots  (131  males  and  19  fe¬ 
males)  ranged  in  age  from  1 8  to  79  with  a  mean  age  of 
42.  Total  flying  hours  ranged  from  6  to  26,500  with 
a  sample  mean  of  1 ,694  hours.  Within  this  sample,  3 1 
were  certified  flight  instructors  (CFI/CFII),  87  held 
private  pilot  certificates,  and  9  were  students.  The 
remaining  subjects  were  certified  as  either  commercial 
or  airline  transport  pilots.  Table  3  contains  the  pilot- 
subjects  self-reported  mean  experience  levels  (0=Never 
flown  into  condition/4=Flown  into  condition  nu¬ 
merous  times). 

Data  Collection  Procedure 

Introductory  information  was  designed  to  be  neu¬ 
tral.  No  reference  was  made  to  research  into  pilot  error 
and  pilot-subjects  were  allowed  to  respond  anony¬ 
mously.  Subjects  were  motivated  to  give  their  per¬ 
sonal  judgments  and  to  avoid  “book”  responses.  It  was 
emphasized  that  the  only  right  answers  were  the  re¬ 
sponses  a  subject  chose  to  make. 

Data  collection  took  place  in  group  settings,  with 
up  to  25  pilots  in  a  group.  Pilots  first  read  the 
instructions  contained  in  part  one  of  the  test  booklet, 
and  then  completed  parts  two  (pilot  information 
form)  and  three  (attribute  experience  ratings).  Pilots 


Table  3.  Mean  Experience  Levels. 


Attribute 

Mean 

- 

Standard 

Deviation 

Terrain 

Water 

1.0 

.95 

Non-mountainous 

3.6 

.95 

Mountainous 

1.4 

1.56 

Precipitation 

Light  rain 

2.4 

1.18 

Moderate  rain 

1.6 

1.35 

Heavy  snow 

.4 

1.00 

Visibility 

1  NM 

1.7 

1.44 

4  NM 

2.7 

1.29 

8  NM 

3.6 

.84 

Ceiling 

800  ft 

1.7 

1.55 

1800  ft 

2.9 

1.23 

4000  ft 

3.6 

.80 

were  then  given  the  first  of  three  sets  of  27  weather 
scenarios  and  (using  the  corresponding  terrain  map) 
asked  to  sort  the  stack  from  least  comfortable  to  most 
comfortable  about  completing  the  flight.  Next,  the 
pilots  were  asked  to  assign  a  O-to-100  “comfort” 
rating  (0=least  comfortable  about  completing  the 
flight/ 100=most  comfortable  about  completing  the 
flight)  to  each  scenario  in  the  set.  Pilots  wrote  their 
rating  on  each  scenario  card  in  a  clearly  marked  box. 
When  pilots  completed  the  ratings  for  a  set  of  weather 
scenarios,  they  were  given  the  next  set  to  complete. 
The  order  of  presentation  of  scenario  sets  to  pilot- 
subjects  was  randomized. 

RESULTS 

Data  analysis  was  designed  to  address  two  primary 
issues.  The  first  focused  on  the  identification  and 
description  of  weights  (worth  functions)  that  pilots 
attribute  to  ceiling,  visibility,  and  precipitation  with 
respect  to  terrain  type.  This  analysis  included  an 
assessment  of  the  relationship  between  pilots’  worth 
functions  and  associated  personal  factors  such  as  age, 
total  flying  hours,  source  of  certification,  etc..  The 
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second  and  more  narrow  issue  involved  data  analysis 
designed  to  detect  the  extent  to  which  noncompensa¬ 
tory  strategies  were  used  by  pilot-subjects.  Analysis 
methodology  and  analysis  results  associated  with  the 
first  issue  (identification  of  worth  functions)  will  be 
described  first. 

Identification  of  Worth  Functions 

Separate  sets  of  analyses  were  conducted  for  each 
terrain  type.  Specifically,  each  pilot-subject’s  worth 
function  (i.e.,  the  relative  emphasis  given  to  ceiling, 
visibility,  and  precipitation  with  respect  to  the  as¬ 
signed  “comfort”  ratings)  was  computed  using  the 
following  regression  equation: 

CF  =  A  +  B  +  C  +  AB  +  AC  +  BC  +  ABC 
where  CF  was  a  vector  of  a  pilot’s  27  weather  scenario 
“comfort”  ratings  for  a  given  type  of  terrain;  and  A,  B, 
and  C  (and  their  interactions)  were  vectors  of 
benchmark  values  (of  the  corresponding  scenarios) 
for  ceiling,  visibility,  and  precipitation,  respectively. 
A  total  of  150  sets  of  regression  weights  (one  set  for 
each  pilot-subject)  were  computed  with  respect  to 
each  terrain  type.  These  individual  sets  of  regression 
weights  were  considered  to  be  a  pilot-subject’s  worth 
function  with  respect  to  the  given  terrain  type. 

values  associated  with  these  weight  sets  were 
relatively  high.  For  mountainous  terrain,  78%  of  the 
150  equations  had  values  equal  to  or  greater  than 
.70.  For  the  non-mountainous  and  over  water  terrain, 
60%  of  the  equations  had  R^  values  equal  to  or  greater 


than  .70.  These  R^  values  indicate  the  degree  to  which 
pilots’  comfort  ratings  are  explained  by  their  worth 
functions  and  scenario  benchmark  values. 

The  150  sets  of  regression  weights  associated  with 
each  terrain  type  were  then  subjected  to  hierarchical 
clustering  (HIER-GRP)  to  identify  groups  of  weight 
sets  differing  significantly  with  respect  to  worth  func¬ 
tion  emphasis.  Within  both  mountainous  and  non- 
mountainous  terrain  types,  four  significantly  different 
policy  groups  were  detected.  Within  the  over  water 
terrain  type,  two  significantly  different  policy  groups 
were  detected.  Tables  4,  5,  and  6  display,  for  the  non- 
mountainous,  mountainous,  and  over  water  scenarios, 
respectively,  the  worth  functions  for  significant  policy 
groups  by  showing  the  average  standard  regression 
weights  which  were  greater  than  or  equal  to  .25.  The 
average  R^,  inter-rater  agreement  (R^),  policy  stabil¬ 
ity  (Rj^i^)j  and  average  comfort  level  (CF)  are  also 
reported  for  each  group  and  terrain. 

Tables  7,  8,  and  9  contain  the  background  data  for 
pilots  in  each  subject  identified  policy  group  for  the 
non-mountainous,  mountainous,  and  overwater  sce¬ 
narios,  respectively.  In  addition,  Appendix  E  lists  the 
results  of  a  series  of  pair  comparisons  of  demographic 
and  comfort  level  values  for  policy  groups  in  each  of 
the  three  scenarios.  Since  the  purpose  of  these  pair 
comparisons  was  to  assist  in  developing  an  under¬ 
standing  of  the  critical  differences  among  these  groups, 
only  those  comparisons  that  were  found  to  be  statis¬ 
tically  significant  are  listed. 


Table  4.  Non-mountainous  Group  Average  Weights. 


A  (Ceiling) 

GP1 

GP2 

GP3 

GP4 

B  (Visibility) 

— 

— 

— 

— 

C  (Precipitation) 

— 

— 

0.51 

— 

AB  (CeitingA/isibililty) 

1.5 

— 

0.34 

— 

AC  (Ceiling/Precipitation) 

0.8 

— 

— 

0.25 

BC  (Visibility/Precipitation) 

— 

— 

— 

— 

ABC  (CeilingA/isibility/Precipitation) 

— 

2.1 

— 

0.37 

Average 

0.79 

0.77 

0.66 

0.58 

Rater  Agreement,  R 

0.57 

0.53 

0.40 

0.35 

Policy  Stability,  R  " 

0.98 

0.98 

0.95 

0.95 

Average  Comfort  tevel 

32.0 

28.0 

50.0 

17.0 

Number  of  subjects 

34.0 

52.0 

27.0 

37.0 

:8 


Table  5.  Mountainous  Group  Average  Weights. 


A  (Ceiling) 

GP1 

GP2 

GP3 

GP4 

B  (Visibility) 

— 

— 

— 

— 

C  (Precipitation) 

— 

— 

— 

— 

AB  (CeilingA/isibililty) 

— 

— 

0.51 

— 

AC  (Ceiling/Precipitation) 

— 

— 

1.1 

0.29 

BC  (Visibility/Precipitation) 

0.91 

— 

— 

— 

ABC  (CeilingA/isibility/Precipitation) 

0.53 

2.3 

— 

0.58 

Average  R^ 

0.83 

0.80 

0.75 

0.60 

Rater  Agreement,  R 

0.49 

0.53 

0.44 

0.35 

Policy  Stability,  R  ” 

0.93 

0.99 

0.96 

0.95 

Average  Comfort  tevel 

36.0 

29.0 

46.0 

17.0 

Number  of  subjects 

14.0 

69.0 

31.0 

36.0 

Table  6.  Over  Water  Group  Average  Weights. 
GP-A  GP-B 

A  (Ceiling)  —  — 

B  (Visibility)  —  — 

C  (Precipitation)  —  — 

AB  (Ceiling/Visibility)  —  — 

AC  (Ceiling/Precipitation)  —  0.98 

BC  (Visibility/Precipitation)  —  0.45 

ABC  (CeilingA/isibility/Precipitation)  1.9  — 

Average 

Rater  Agreement,  R 
Policy  Stability,  R 
Average  Comfort  tevel 
Number  of  subjects 


0.69 

0.67 

0.43 

0.35 

0.99 

0.95 

27.0 

40.0 

12.0 

38.0 
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Table  7.  Mean  Demographic  Variables  and  Comfort  Levels  for  Non- 


Mountainous  Scenario  Groups. 


Background  Variable 

Non-mountainous  Groups 

GP2 

GP3 

GP4 

J-N 

J-B 

S-C 

Number  of  Subjects 

34 

52 

27 

37 

Flying  History 

■M 

Total  Hours 

1869 

1679 

Last  90  Days 

mm 

47 

25 

Average  Age 

44 

38 

46 

Percent  Instrument  Rated 

53% 

38% 

48% 

42% 

Percent  Owning  Aircraft 

32% 

44% 

19% 

41% 

Percent  Female 

12% 

17% 

4% 

14% 

Mean  Terrain  Experience 

3.50 

3.65 

3.41 

3.49 

Mean  Precipitation  Experience 

Light  Rain 

2.62 

2.50 

2.41 

2.11 

Moderate  Rain 

1.79 

1.71 

1.63 

1.27 

Heavy  Snow 

.38 

.33 

.67 

.38 

Mean  Visibility  Experience 

1  NM 

1.62 

1.79 

1.85 

1.38 

4NM 

2.74 

2.92 

3.00 

2.24 

8NM 

3.38 

3.77 

3.70 

3.30 

Mean  Ceiling  Experience 

800  ft 

1.65 

1.75 

1.78 

1.62 

1800  ft 

2.88 

3.13 

2.93 

2.46 

4000  ft 

3.56 

3.81 

3.67 

3.38 

Percent  Using  Weather  Source 

FAA/FSS 

74% 

66% 

56% 

68% 

TV/Radio 

33% 

32% 

35% 

26% 

Other 

0% 

12% 

7% 

13% 

None 

0% 

4% 

3% 

1% 

Percent  Filing  Flight  Plan* 

58% 

55% 

45% 

57% 

Percent  Certificated  Held 

PP 

53% 

58% 

56% 

70% 

COM/ATP 

32% 

37% 

22% 

24% 

CFI/CFII 

21% 

21% 

26% 

16% 

Percent  Flying 

OiO/ 

WSM 

Job-employed  as  Pilot 

Z  1  /o 

4  4  0/ 

17% 

3% 

Business/Work  Related 

1  1  To 

C4  0/ 

7% 

6% 

Pleasure 

blvo 

74% 

60% 

85% 

Average  Comfort  Level 

32 

28 

50 

17 

Average  R^ 

.79 

.77 

.66 

.58 

*  Letter  choices  were  converted  to  a  O-to-100  range:  a=100,  b=75,  c=50,  d=25,  e=0. 
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Table  8.  Mean  Demographic  Variables  and  Comfort  Levels  for 
Mountainous  Scenario  Groups. 


Background  Variable 


Mountainous  Groups 


GP1 

GP2 

GP3 

GP4 

S-N 

J-N 

J-B 

S-C 

Number  of  Subjects 

14 

69 

31 

36 

Flying  History 

Total  Hours 

1969 

1963 

561 

2140 

Last  90  Days 

47 

49 

34 

21 

Average  Age 

51 

41 

34 

48 

Percent  Instrument  Rated 

51% 

36% 

50% 

39% 

Percent  Owning  Aircraft 

43% 

39% 

16% 

44% 

Percent  Female 

7% 

6% 

25% 

Mean  Terrain  Experience 

1.57 

1.10 

1,36 

Mean  Precipitation  Experience 
Light  Rain 
Moderate  Rain 
Heavy  Snow 


Mean  Visibility  Experience 
1  NM 
4NM 
8NM 


Mean  Ceiling  Experience 
800  ft 
1800  ft 
4000  ft 


Percent  Using  Weather  Source 
FAA/FSS 
TV/Radio 
Other 
None 


Percent  Filing  Flight  Plan* 


Percent  Certificates  Held 
PP 

COM/ATP 

CFI/CFII 


Percent  Flying 
Job-employed  as  Pilot 
Business/Work  Related 
Pleasure 


Average  Comfort  Level 


Average  R^ 


Letter  choices  were  converted  to  a  O-to-100  range;  a=100,  b=75,  c=50,  d=25,  e=0. 
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Table  9.  Mean  Demographic  Variables  and  Comfort  Levels 
for  over-Water  Scenario  Groups. 


Background  Variable 


Number  of  Subjects 


Flying  History 

Total  Hours 
Last  90  Days 


Average  Age 


Percent  Instrument  Rated 


Percent  Owning  Aircraft 


Percent  Female 


Mean  Terrain  Experience 


Mean  Precipitation  Experience 
Light  Rain 
Moderate  Rain 
Heavy  Snow 


Mean  Visibility  Experience 
1  NM 
4NM 
8NM 


Over-Water  Groups 


GP-A 

T-N 


Percent  Using  Weather  Source 
FAA/FSS 
TV/Radio 
Other 
None 


Percent  Filing  Flight  Plan* 


Percent  Certificates  Held 
PP 

COM/ATP 

CFI/CFII 


Percent  Flying 

Job-employed  as  Pilot 
BusinessAA/ork  Related 
Pleasure 


Average  Comfort  Level 


Average  R^ 


Letter  choices  were  converted  to  a  O-to-100  range:  a=100,  b=75,  c=50,  d=25,  e=0. 
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With  respect  to  non-mountainous  terrain  (Table 
4),  four  worth  functions  could  be  interpreted.  The 
most  obvious  of  these  were  those  associated  with  GPl 
and  GP2,  which  we  have  chosen  to  designate  as  the 
Senior-Nominal  Group  and  Junior-Nominal  Group, 
respectively,  based  upon  an  inspection  of  the  signifi¬ 
cant  differences  among  demographic  characteristics 
and  comfort  levels  of  the  groups,  as  shown  in  Table  7 
and  Appendix  E.  The  worth  function  associated  with 
GPl  seemed  to  be  related  to  pilots  who  emphasized 
the  interactive  relationship  of  ceiling  and  visibility  in 
their  assessment  of  flight  completion  comfort.  The 
function  associated  with  GP2  appeared  to  represent 
pilots  who  interactively  related  ceiling,  visibility,  and 
precipitation.  The  remaining  functions  for  GP3  Qun- 
ior-Bold)  and  GP4  (Senior-Cautious)  were  less  dis¬ 
tinct  (i.e.,  did  not  have  standard  weights  greater  or 
equal  to  1.0).  All  mountainous  terrain  worth  func¬ 
tions  (Table  5)  appeared  to  involve  the  interrelation¬ 
ship  of  pairs  of  weather  attributes  or  all  three  weather 
attributes.  With  respect  to  over-water  terrain  (Table 
6),  there  were  two  relatively  distinct  functions.  The 
function  for  GP-A  (Total-Nominal)  was  similar  to  the 
previously  cited  GP2  (Junior-Nominal)  functions. 
Significant  differences  in  background  variable  values 
among  non-mountainous,  mountainous,  and  over¬ 
water  groups  are  listed  in  Appendix  E. 

Noncompensatory  Strategy  Detection 

To  determine  the  extent  to  which  noncompensatory 
strategies  were  used  by  the  subject-pilots  in  assigning 
weather  scenario  comfort  ratings,  the  relationships  (cor¬ 
relations)  between  comfort  ratings  and  sets  of  compen¬ 
satory  and  noncompensatory  models  were  examined  and 
compared.  Specifically,  the  standardized  scenario  bench¬ 
mark  ratings  (which  had  been  standardized  to  a  mean  of 
5.0  and  a  standard  deviation  of  1 .0  for  all  scenarios)  were 
first  transformed  (using  sets  of  transformations  designed 
to  simulate  compensatory  and  noncompensatory  poli¬ 
cies),  and  then  correlated  with  pilot-subjects*  scenario 
mean  comfort  ratings  for  each  terrain  type.  The  compen¬ 
satory  model  set  consisted  of  additive,  multiplicative, 
and  worst-factor  cut-off  models.  The  noncompensatory 
set  consisted  of  sets  of  single-factor,  continuous,  and  cut¬ 
off  models  (one  set  for  each  factor  —  ceiling,  visibility, 
and  precipitation). 


Within  the  compensatory  model  set,  the  additive 
model  summed  the  three  benchmark  factors,  and 
implied  equal  weights  for  the  three  factors.  The  mul¬ 
tiplicative  models  (both  two  and  three  factor  models) 
cross-multiplied  benchmark  values,  implied  a  policy 
in  which  one  or  more  factors  may  introduce  a  damp¬ 
ening  effect  which  can  overpower  the  level  of  the  other 
factors.  The  worst-factor  cut-off  models  used  the 
lowest  of  the  three  benchmark  values,  and  implied  a 
policy  in  which  the  poorest  of  the  three  factors  became 
the  only  focus.  Within  the  noncompensatory  set, 
models  representing  policies  associated  with  continu¬ 
ous  single  factors  and  single-factor  cutoffs  (ceiling, 
visibility,  or  precipitation)  were  used. 

Table  10  lists  the  compensatory  and  noncompensa¬ 
tory  models  used,  and  their  respective  correlations 
with  pilot-subject  mean  comfort  ratings.  It  may  be 
seen  that  in  almost  all  instances,  the  highest  correla¬ 
tions  (across  terrain  types)  were  always  associated  with 
the  compensatory  models.  Of  these  compensatory 
models,  the  three-factor  additive  and  multiplicative 
models  consistently  showed  the  strongest  relation¬ 
ships  with  assigned  comfort  ratings.  Overall  results 
tend  to  support  the  contention  that  compensatory 
strategies  were  widely  used  by  pilot  subjects. 

DISCUSSION  AND  CONCLUSIONS 

First,  let  us  note  that  the  results  of  this  study  must 
be  considered  tentative  in  that  the  worth  functions 
were  derived  from  a  subject-pilot  sample  primarily 
experienced  in  flying  over  non-mountainous  terrain. 
Because  of  the  terrains  used  and  the  experience  base  of 
the  pilots  sampled,  this  study  may  produce  unstable 
weights  for  the  “water”  and  “mountain”  terrains  as 
experience  levels  were  very  low.  In  particular,  low 
experience  levels  over  water  may  account  for  the  fact 
that  only  two  policies  were  detected.  Additional  stud¬ 
ies  (in  other  parts  of  the  country  using  subjects  with 
higher  levels  of  mountainous  and  over-water  flying 
experience)  will  be  required  to  assess  the  stability  of 
the  obtained  worth  functions.  These  studies  will  also 
allow  us  to  evaluate  the  degree  to  which  the  compen¬ 
satory  and  noncompensatory  models  are  used  by  pilot 
in  these  settings. 
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Table  10.  Comparison  of  Compensatory  and  Non- 
Compensatory  Models. 


Models  Used 

Obtained  Correlation  (r) 

Non»mountainous 

Mountainous 

Water 

Compensatory  Models 

A  4-  B  +  C  (Additive) 

.94 

.93 

.83 

A  *  B  *  C  (Multiplicative) 

.96 

.96 

.89 

A  *  B  (Multiplicative) 

.86 

.84 

.65 

A  *  C  (Multiplicative) 

.76 

.82 

.79 

B  *  C  (Multiplicative) 

.79 

.86 

.73 

Worst  Factor  Models 

Worst  Factor  (X  >  5.0) 

.69 

.71 

.54 

Worst  Factor  (X  >  4.5) 

.80 

.82 

.78 

Worst  Factor  (X  >  4.0) 

.76 

.79 

.74 

Noncompensatory  Models 

A 

.48 

.53 

.46 

A(X>5.0) 

.44 

.55 

.46 

A(X>4.5) 

.38 

.42 

.35 

A(X>4.0) 

.38 

.42 

.35 

B 

.67 

.62 

.48  ■ 

B  (X>  5.0) 

.56 

.53 

.41 

B(X>4.5) 

.62 

.46 

.46 

B(X>4.0) 

.62 

.46 

.46 

C 

.60 

.73 

.62 

C<X>5.0) 

.60 

.67 

.57 

C(X>4.5) 

.53 

.56 

.50 

C(X>4.0) 

.35 

.55 

.41 

A  =  Ceiling 
B  =  Visibility 
C  =  Precipitation 
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Subject  to  that  caveat,  the  obtained  analysis  results 
were  sufficiently  clear  to  allow  the  identification  (at 
least  tentatively)  of  distinctly  different  worth  func¬ 
tions.  Specifically,  four  different  worth  functions 
were  identified  and  characterized  for  non-mountain- 
ous  and  mountainous  terrain  types.  Two  different 
worth  functions  were  identified  and  characterized  for 
over-water  terrain.  Pilot  use  of  weather  information 
was  consistent  with  expert  assessmen  t  of  the  safety  risk 
associated  with  VFR  flight  under  varying  ceiling, 
visibility,  and  precipitation  conditions  over  non- 
mountainous,  mountainous,  and  water  terrain.  How¬ 
ever,  while  use  is  consistent,  pilots  vary  in  their 
expressed  degree  of  comfort  in  flying  over  the  differ¬ 
ent  terrain  under  different  weather  conditions.  These 
differences  in  comfort  level  may  occur,  in  part  at  least, 
because  of: 

•  Differences  of  understanding  of  the  risks  associated 

with  flying  under  differing  above-minimum  levels 
of  visibility,  ceiling,  and  precipitation  and  how 
terrain  type  impacts  use  of  such  information. 

•  Differences  in  their  self-assessment  and  perception 

of  their  ability  and/or  skill  in  flying  under  the 
varying  conditions. 

Training  or  other  interventions  addressing  risk 
assessment  and  self-perception  would  appear  to  be 
appropriate.  The  use  of  training  directed  at  the  Five 
Hazardous  Thought  Patterns  is  one  method  of  ad¬ 
dressing  self  perception.  T  raining  directed  at  improv¬ 
ing  pilot  understanding  of  the  effects  of  terrain-type 
on  interaction  of  meteorological  conditions  is  another 
potential  source.  Increased  emphasis  on  risk  assess¬ 
ment  and  self  perception  training  and  exercises  in 
initial  training  and  in  subsequent  seminars  (FAA- 
sponsored  and  otherwise)  would  seem  to  be  war¬ 
ranted.  Careful  consideration  must  be  given,  however, 
to  the  desired  outcome  of  this  training.  The  present 
study  has  shown  that  many  pilots  use  a  compensatory 
model  for  evaluating  weather  information,  however 
no  data  have  been  collected  to  demonstrate  that  the 
use  of  a  compensatory  model  is  appropriate  for  all  or 
even  the  majority  of  pilots. 

Consider  the  implications  ofa  compensatory  model 
in  which  a  high  value  for  one  variable  (say,  visibility) 
can  compensate  for  a  low  value  on  another  variable 


(say,  ceiling)  in  making  the  determination  of  the 
suitability  of  the  weather  conditions  for  flight.  From 
the  standpoint  of  information  processing,  this  maybe 
an  efficient  use  of  information,  however  it  may  also 
lead  pilots  to  make  poor  decisions  under  some  cir¬ 
cumstances.  It  may  be  argued  that  in  many  situations 
(for  example,  in  mountain  flying)  a  noncompensatory 
model  should  be  used.  One  such  model  might  set  a 
minimum  value  for  ceiling  values  (sufficient  to  clear 
all  mountain  peaks,  for  example)  which  must  be  met, 
regardless  of  the  visibility.  That  is,  in  such  a  model, 
having  a  very  high  visibility  does  not  compensate  for 
having  a  low  ceiling.  Under  those  circumstances, 
training  inexperienced  pilots  to  use  the  compensatory 
model  used  by  experts  might  be  inappropriate  and 
could,  in  fact,  lead  to  a  decrease  in  safety.  Indeed,  the 
dominance  of  the  compensatory  models  held  by  central- 
Texas  pilots  might  partially  account  for  accidents  involv¬ 
ing  these  pilots  when  they  fly  into  mountainous  areas. 

Thus,  the  use  of  a  compensatory  model,  even  though 
it  is  the  favored  mode,  may  be  hazardous  under  some 
circumstances.  Training  might  therefore  be  devel¬ 
oped  that  would  allow  pilots  to  assess  their  personal 
decision  making  model,  and  which  would  train  them 
in  the  use  of  non-compensatory  models  where  such 
models  might  reduce  risk. 

In  summary,  this  study  found: 

•It  is  possible  to  capture  reliably  pilots*  “comfort” 
policies  in  terms  of  emphasis  upon  terrain,  ceiling, 
visibility,  and  precipitation  using  a  linear  model¬ 
ing  approach. 

•Pilots  use  compensatory  judgment  rules  with  so¬ 
phisticated  interactions. 

•Pilots  generally  agree  on  how  to  prioritize  the 
safety  implications  of  terrain/weather  conditions. 
•Pilots’  policies  tend  to  fall  cleanly  into  stable 
policy  groups  although  the  individual  pilots  who 
hold  those  views  switch  groups  according  to  ter¬ 
rain. 

•  Pilots’  policies  vary  based  upon: 

1.  General  comfort  level 

2.  A  mixture  of  age  and  general  experience 

3.  Number  of  hours  flown  in  last  90  days 

4.  Reasons  for  flying  (employed  as  pilot  versus 
pleasure  flying) 
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APPENDIX  A 


DATA  COLLECTION  PACKAGE 


Rate  your  comfort  level  about  proceding  given  the  following  weather 
update  (100  =  The  MOST  comfortable;  0  =  the  LEAST  comfortable): 


ENROUTE:  A  weak  cold  front  has  stalled  along  a  line  from  Sturgeon 
Bay  to  Manistique.  Ceilings  are  reported  at  5000  ft  with  visibility  of  1 14  NM 
and  light  rain. 

ROUTE  1 

Card  Number  801 

Rate  your  comfort  level  about  proceding  given  the  following  weather 
update  (100  =  The  MOST  comfortable;  0  =  the  LEAST  comfortable): 


ENROUTE:  A  weak  cold  front  has  stalled  along  a  line  running  west  from 
Amarillo  into  New  Mexico..  Ceilings  are  reported  at  4000  ft  with  visibility  of  1 
NM  and  light  rain. 

ROUTE  2 

Card  Number  60 1 

Rate  your  comfort  level  about  proceding  given  the  following  weather 
update  (100  =  The  MOST  comfortable;  0  =  the  LEAST  comfortable): 


ENROUTE:  A  weak  cold  front  has  stalled  along  a  running  from  Santa 
Fe  northeast  into  Colorado.  Ceilings  are  reported  at  600  ft  with  visibility  of  5 
NM  and  freezing  rain. 

ROUTE  3 

Card  Number  701 


A3 


Weather  Combinations:  Route  1 —  Delta  County  to  City  County. 


Scenario 

Ceiling 

Visibility 

Precipitation 

800 

4000 

7 

moderate  rain 

801 

5000 

1.5 

light  rain 

802 

4000 

4 

none 

803 

5000 

8+ 

none 

804 

2000 

3 

none 

805 

1800 

3 

heavy  rain 

806 

1800 

7 

light  snow 

807 

3500 

1 

heavy  rain 

808 

CD 

O 

O 

1.5 

moderate  rain 

809 

1800 

7 

moderate  rain 

810 

800 

1.5 

none 

811 

1600 

8 

none 

812 

1600 

3 

heavy  snow 

813 

4000 

5 

light  snow 

814 

900 

0.5 

heavy  snow 

815 

1600 

1.5 

heavy  rain 

816 

5000 

0.5 

moderate  rain 

817 

600 

8 

moderate  rain 

818 

600 

7 

none 

819 

1600 

1.5 

moderate  snow 

820 

5000 

8+ 

freezing  rain 

821 

900 

5  . 

none 

822 

800 

4 

moderate  rain 

823 

1600 

4 

moderate  snow 

824 

1800 

5 

moderate  rain 

825 

1800 

0.5 

none 

826 

900 

4 

heavy  rain 

A4 


Weather  Combinations:  Route  2—  Dalhart  Airport  to 
Hereford  Airport. 


Scenario 

Ceiling 

Visibility 

Precipitation 

600 

5000 

8 

light  snow 

601 

4000 

1 

light  ram 

602 

1600 

3 

heavy  snow 

603 

4000 

CO 

light  rain 

604 

2000 

7 

light  snow 

605 

4000 

8+ 

light  rain 

606 

3500 

0.5 

heavy  rain 

607 

1800 

0.5 

moderate  snow 

608 

600 

0.5 

light  snow 

609 

1800 

4 

none 

610 

800 

8+ 

light  snow 

611 

800 

8 

none 

612 

1800 

5 

moderate  rain 

613 

600 

8 

moderate  rain 

614 

900 

4 

heavy  rain 

615 

5000 

1.5 

moderate  rain 

616 

1800 

7 

moderate  rain 

617 

900 

5 

light  snow 

618 

900 

0.5 

heavy  rain 

619 

2000 

1.5 

none 

620 

1800 

3 

heavy  rain 

621 

2000 

0.5 

freezing  rain 

622 

1600 

4 

moderate  snow 

623 

800 

3 

none 

624 

900 

1.5 

light  rain 

625 

2000 

8+ 

light  rain 

626 

4000 

5 

light  snow 

A5 


Weather  Combinations:  Route  3 —  San  Juan  Pueblo  Airport 

to  Las  Vegas  Airport. 


Scenario 

Ceiling 

Visibility 

Precipitation 

700 

5000 

5 

none 

701 

600 

5 

freezing  rain 

702 

3500 

8 

moderate  rain 

703 

900 

1 

heavy  snow 

704 

2000 

0.5 

moderate  rain 

705 

900 

4 

heavy  rain 

706 

4000 

8+ 

none 

707 

600 

3 

moderate  snow 

708 

1800 

7 

moderate  rain 

709 

800 

1 

none 

710 

4000 

0.5 

heavy  snOw 

711 

600 

1 

moderate  snow 

712 

4000 

4 

light  snow 

713 

3500 

1 

light  snow 

714 

1600 

3 

heavy  rain 

715 

2000 

8+ 

light  rain 

716 

2000 

1.5 

none 

717 

1600 

1 

freezing  rain 

718 

600 

8 

moderate  rain 

719 

3500 

3 

freezing  rain 

720 

2000 

5 

light  snow 

721 

900 

5 

light  rain 

722 

900 

8 

light  rain 

723 

1600 

3 

heavy  snow 

724 

2000 

j 

4 

none 

725 

5000  I 

1.5 

light  rain 

726 

1800 

5 

moderate  rain 

A6 
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radio  is  broken  and  you  do  not  have  a  transponder.  It  is  1 1 :00  AM  local  time,  you  are 
topped  off  with  avgas,  are  carrying  one  8  pound  suitcase,  and  receive  the  following 
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HEREFORD  WEATHER:  Clear,  visibilty  1 0+NM,  wind  240/1 0,  temperature  48, 
dew  point  40,  altimeter  setting  29.90 

ENROUTE  WEATBDER:  See  ENROUTE  weather  scenario  list. 


DALHART  COUNTY  WEATHER;  9,000  overcast,  visibility  5  NM,  wind  350/10, 
temperature  40,  dewpoint  32,  altimeter  setting  30.01 


A8 


MAG  HEADING 


296 


MAG  HEADING 

233 

DISTANCE 

22NM 

TIME  (NO  wind) 

-  ■■■■  »  V  N 

13Min 

M-'l 

-5^^^’V«J,JjSa- 


DISTANCE 

15NM 

j  TIME  (NO  wind) 

11  Min 

^4’  V'-j 


^'V*' 


il-i 


tm 


yAt%%  .. 


m^4 


MAG  HEADING 

323 

DISTANCE 

25NM 

TIME  (NO  wind) 

20Mm 

You  are  solo  cnroutc  to  the  Delta  County  Airport  (ESG)  in  Escanaba,  Michigan 
and  plan  to  fly  VFR  following  the  route  shown  on  this  sectional  labeled 
ROUTE!.  You  have  made  a  fuel  stop  at  the  City  County  Airport 

(3D4)  east  of  Frankfort,  Michigan.  Your  Cessna  1 72  is  fully  equipped 
for  over-water  flight  and  working  perfectly  except  that  the  navigation 
radio  is  broken  and  you  do  not  have  a  transponder.  It  is  1!  :00  AM  local  time, 
you  arc  topped  off  with  avgas,  arc  carrying  one  8  pound  suitcase,  find 
that  IR  609  and  IR  61 0  are  not  in  use,  and  receive  the  following  weather  report: 

FRANKFORT  WEATHER;  Clear,  visibilty  1 0+NM,  wind  240/10,  temperature  48 
dewpoint  40,  altimeter  setting  29.90 

ENROUTEWEIATHER:  See  ENROUTE  weather  cards. 

DELTA  COUNTY  WEATHER:  9,000  overcast,  visibility  5  NM,  wind  350/10, 
temperature  40,  dew  point  32,  altimeter  setting  30.0 1 


APPENDIX  B 


PILOT  INFORMATION  FORM 
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Table  B-1 .  Pilot  Demographic  Information. 


Table  B-2.  Flying  Categories 


Count 

Percent 

CERTIFICATES  HELD: 

Private  Pilot - 

....89-- 

— 59% 

Commercial  Pilot - 

....35.... 

— 23% 

Airline  Transport  Pilot - 

—  12  — 

-  8% 

Certified  Flight  Instructor  - 

....13.... 

- 9% 

Certified  Flight  Instructor 

Instrument - 

—  24  — 

- 16% 

INSTRUMENT  RATED: 

Yes - 

....  67  — - 

- 45% 

No - 

..-83  — 

- 55% 

STATE  OF  RESIDENCE 

Colorado - 

Maryland - 

New  Mexico - 

T  exas - 

Wisconsin - 

Other - 

AGE': 

16  -  20 - -—10 - 7% 

21  -  40  - 59  39% 

41  -  50  - 28  19% 

51  .  60  - 30 - 20% 

61  -  70 - 18 - 12% 

71  + - 5 - 30/0 

SEX: 

Female - 1 9 - 13% 

Male -  130  87% 

Blank - 1  1% 


Count  Percent 

PERCENT  OF  FLYING  AS: 

PRIMARY  JOB-EMPLOYED 
AS  A  PILOT 


0% - 

1  -  20% - 

21  -  40%  — 
4l  .  60%  _ 

—  120 

- 2 

- 1 

- 3 

_ A 

- - - 80% 

-  1% 

_ _  0  0/n 

“X  \J\J  /u 

-  80%  _ 

z.  /o 

...  ZO/n 

81  -  100%- 

TC 

- 20 

- 13% 

PERCENT  OF  FLYING  AS: 

BUSINESS  SUPPORT- 
WORK  RELATED 

Qo/o - 116 - 77% 

1  -  20% - 15 - 10% 

21  -  40% - 3 - 2% 

41  -  60% - 9 - 6% 

61  -  80% - 3 - 2% 

81  -  100% - 4 - 3% 

PERCENT  OF  FLYING  AS: 

PLEASURE 

QO/o  - 1  5  . 1 0% 

1  -  20% - 20 - 13% 

21  -  40% - 5 - 3% 

41  -  60% - 7 - 5% 

61  -  80% - 6 - 4% 

81  -  100% - 97  - 65% 


...1 -  10/0 

-„1 -  10/0 

1 -  1 0/0 

144 - 960/0 

...  1 -  1 0/0 

...2 -  1% 


ENGINE  RATING: 

Single -  108  72% 

Multi - 37 - 25% 

Other - 5 - 3% 


MEDICAL  CERTIFICATE: 


Class  1 - 

- -30..- 

- 20% 

Class  2 - 

- 49.... 

- 33% 

Class  3 - 

- 69-- 

—  46% 

EVER  BEEN  A  MILITARY  PILOT? 

No -  135  - 90% 

Yes - 15 - 10% 

Note  1  :  Age  Mean  =  42.64;  S.D.  =  15.44 


Table  B-3.  Flight  Plan  Use. 


HOW  OFTEN  DO  YOU  FILE  A 
FLIGHT  PLAN  WITH  THE  FAA? 


Count 

Percent 

All  of  the  time - 

„22 - 

— 150/0 

Most  of  the  time - 

„44 - 

— 29% 

Some  of  the  time - 

-43 - 

— 29% 

Seldom - 

-23  - 

— 150/0 

Never - 

...  9  - 

- 6% 

(Invalid/No  response) 

...  9  - 

- 6% 

Table  B-4.  Weather  information  sources 


PERCENT  USE  OF: 

FAA/FSS 

Count 

Percent 

QO/o  - 

- 14 - 

—  9% 

1  -  20% - 

- 16 - 

— 110/0 

21  -  30%  - 

- 9  - 

- 6% 

41  -  60% - 

- 19 - 

— 13% 

61  -  80% - 

- 15 - 

- 10% 

81  -  100% - 

- 77 - 

—  51% 

PERCENT  USE  OF:  TV/RADIO 

QO/o  - 

- 46 - 

- 31% 

1  -  20% - 

- 38 - 

- 25% 

21  -  40% - 

- 12 - 

- 8% 

41  -  60% - 

- 22 - 

- 150/0 

61  -  80% - 

- 11  - 

- 70/0 

81  -  100% - 

- 21  - 

- 140/0 

PERCENT  USE  OF: 

OTHER 

QO/o  - 

- 110 - 

- 730/0 

1  -  20% - 

- 17 - 

- 110/0 

21  -  40% - 

- 6 - 

- 40/0 

41  -  60% - 

- 5  - 

- 30/0 

61  -  80% - 

- 6 - 

- 40/0 

81  -  100% - 

- 6 - 

- 4% 

PERCENT  USE  OF: 

NONE 

QO/o  - - 

- 139 - 

— 930/0 

1  -  20% - 

- 7 - 

- 50/0 

21  -  40% - 

- 1  - 

- 10/0 

41  -  60% - 

- 1  - 

- 10/0 

61  -  80% - 

- 0  - 

- Oo/o 

81  -  100% - 

- 2  - 

- 10/0 

Table  B-5.  Flying  History -Total  Hours. 


TOTAL  FLYING  HOURS 


Count 

Percent 

0  -  99 - 

- 20  — 

— 130/0 

100  -  299  — 

- 48 - 

— 32% 

300  -  599 - 

- 24 - 

— 160/0 

600  -  899 - 

- 13 - 

—  90/0 

900  -  1199  — 

- 7  - 

—  50/0 

1200  -  1499  - 

- 5  - - 

—  30/0 

1500  -  1799  - 

- 4  - 

—  30/0 

1800  -  2099  - 

- 4  - - 

—  30/0 

2100  -  2399  - 

- 1  - 

— 10/0 

2400  -  2699  - 

-  1  - 

. 1% 

2700  -  2999  - 

- 2  - 

. 1% 

3000  -  30000 

- 21  - 

- 14% 

Mean: - 

1716.89 

S.D. - 

3742.17 

DAY  HOURS 


0  -  99 - 

- 29 - 

- 19% 

100  -  299 - 

- 44 - 

- 29% 

300  -  599 - 

- 25 - 

- 170/0 

600  -  899 - 

- 9  - 

- 6% 

900  -  1199  — 

- 7 - 

- 50/0 

1200  -  1499  - 

- 7 - 

- 50/0 

1500  -  1799  - 

- 4 - 

- 30/0 

1800  -  2099  - 

- 1  - 

- 10/0 

2100  -  2399  - 

- 1  - 

- 10/0 

2400  -  2699  - 

- 2  - 

- 10/0 

2700  -  2999  - 

- 2  - 

- 10/0 

3000  -  30000 

- 19 - 

- 13% 

Mean: - 

1418.14 

S.D. - 

3036.35 

NIGHT  HOURS 


0 - 

- 4  - 

- 30/0 

1  .  19 - 

- 53 - 

- 35% 

20  -  49  - 

- 31  - 

- 21% 

50  .  99 - 

- 16 - 

- 110/0 

100  -  299 - 

- 24 - 

- 16% 

300  -  599 - 

- 7 - 

- 50/0 

600  -  899 - 

- 3  - 

- 2% 

900  -  1199  — 

- 3  - 

- 2% 

1200  -  1499  - 

- 2  - 

- 10/0 

1500  -  15000 

- 7  - 

- 5% 

Mean: - 

-293.01 

S.D. - 

1054.43 

B4 


Table  B-5.  (Continued) 


ACTUAL  INSTRUMENT  HOURS 


0 - 

- 47 - 

- 31% 

1  .  19 - 

- 49 - 

- 33% 

20  -  49  - 

- 12 - 

- 8% 

50  -  99  - 

- 14 - 

- 9% 

100  -  299 - 

- 12 - 

- 8% 

300  -  599 - 

- 4 - 

- 3% 

600  -  899 - 

- 2  - 

-  lO/o 

900  -  1199  — 

- 3  - 

- 2% 

1200  -  1499  - 

- 2  - 

- 1  o/o 

1500  -  15000- 

- 5  - 

- 3% 

Mean: - 

-220.09 

S.D. - 

-995.52 

ACTUAL  PLUS  HOOD  HOURS 


0 - 

1  -  19 - 

- 24 - 

- 38 - 

- 160/0 

- 25% 

20  -  49  - 

- 22 - 

- 15% 

50  -  99  - 

- 31  - 

- 210/0 

100  -  299 - 

- 16 - 

- lio/o 

300  -  599 - 

- 7 - 

- - 5% 

600  -  899 - 

- 2  - 

-  lo/o 

900  -  1199  — 

- 0  - 

- Oo/o 

1200  -  1499  - 

- 2  - 

-  lo/o 

1500  -  15000 

- 8  - 

- 50/0 

Mean: - 

-276.26 

S.D. - 

1181.99 

NIGHT  INSTRUMENT  HOURS 


0 - 

- 72 - 

- 480/0 

1  .  19 - 

- 6 - 

- 31% 

20  -  49  - 

- 13 - 

- 9% 

50  -  99  - 

- 3  - 

—  2% 

100  -  299 - 

- 10 - 

- 70/0 

300  -  599 - 

- 2  - 

- 10/0 

600  -  899 - 

- 1  - 

- 10/0 

900  -  1199 - 

- 1  - 

- 10/0 

1200  -  1499  — 

- 0 - 

- QO/o 

1500  -  15000- 

- 2 - 

-  10/0 

Mean: - 

105.28 

S.D. - 

767.16 

TOTAL  INSTRUMENT  APPROACHES  FLOWN 


0 - 

1  .  24 - 

25  -  49  - 

SO  00 

- 60 - 

- 16 - 

- 16 - 

1  ^ 

- 400/0 

- 110/0 

- 110/0 

1  10/. 

-  yy 

100  -  299  — 

- 22  - 

- 150/0 

300  -  599  — - 

- 6 - 

- 40/0 

600  -  899  — 

- -  1  - 

- 10/0 

900  -  9999  - 

- 13 - 

- 90/0 

Mean: - 

-291.67 

S.D. - 

-  964.28 

Table  B-6.  Flying  Experience  in  the  Last 
90  Days. 


TOTAL  HOURS 

Count 

Percent 

0 - 

- 1 2 . 

. 8% 

1  .  24 - 

-  70  - 

—  47% 

25  -  49  - - 

- 36 - 

—  24% 

50  -  99 - 

— 13  — 

—  9% 

100  -  299 - 

-  18  - 

—  1 2% 

300  -  9999  - 

-  1  - 

-  1% 

Mean: - 

-39.45 

S.D. - 

-54.69 

DAY  HOURS 

0 - 

- 15 - 

-—10% 

1  .  24 - 

- 74 - 

-—49% 

25  -  49  - 

- 34 - 

—  23% 

50  -  99 - 

- 12 - 

- 8% 

100  -  299 - 

- 14 - 

- 9% 

300  -  9999  - 

- 1  - 

- —  10/0 

Mean: - 

-36.95 

S.D. - 

-54.52 

NIGHT  HOURS 

0 - 

- 69 - 

—  46% 

1  .  24 - 

- 79 - 

—  53% 

25  -  49  - 

-  1  - 

-  1% 

50  -  99 - 

- 0  - 

- 0% 

100  -  299 - 

-  1  - 

- 1% 

300  -  9999 - 

-  0  - 

- 0% 

Mean: - 

- 3.99 

S.D. - 

- 9.41 

ACTUAL  INSTRUMENT  HOURS 

0 - 

- 87 - 

—  58% 

1  .  24 - 

- 58 - 

— 39% 

25  -  49  - 

- 3  - 

- 2% 

50  -  99 - 

-  2 - 

-  1% 

100  -  9999  — 

- 0  - 

- 0% 

Mean: - 

—3.55 

S.D. - 

-—9.39 
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Table  B-6,  (Continued) 

TOTAL  INSTRUMENT  HOURS 

0 - „81  54% 

1  _  24 - 62  41% 

25  -  49  -  6  4% 

50  -  99 -  1  lo/o 

100  -  9999 -  0  0% 

Mean: - 4.94 

S.D. - 8.98 


Table  B-7.  Characteristics  of  Aircraft 
Most  Frequently  Flown. 

NUMBER  OF  ENGINES 

Count  Percent 

One  Engine - 136 - 91% 

Two  Engines -  8 - 5% 

(Other/No  Response)  —  6  - 4% 

IFR  CAPABLE? 


ACTUAL  NIGHT  INSTRUMENT  HOURS 


0 - 

- 118  — ■ 

- 79% 

1  -  24 - 

-  31  — - 

- 21% 

25 . 49  - 

-  0  — - 

- 0% 

50  -  99  - 

-  1 

-  1% 

100  -  9999  — 

- 0  — - 

- 0% 

Mean: - 

— 1.06 

S.D. - 

— 6.61 

Yes - 

119 - 

- 79% 

No - 

..  30 - 

- 20% 

(Other/No  Response) 

—  1  - 

-  lo/o 

AUTO-PILOT  EQUIPPED? 

Yes - 

-40 - 

- 27% 

No - 

108 - 

- 72% 

(Other/No  Response) 

—  2  - 

-  lO/o 

NUMBER  OF  INSTRUMENT  APPROACHES 
FLOWN 

0 - 81  54% 

1  .  24 - 54  36% 

25  -  49  - 11  7% 

50  -  99  - 3  2% 

100  -  299 -  1  1% 

300  -  9999 -  0  0% 

Mean: - 7.02 

S.D. - 13.93 


WEATHER  RADAR  EQUIPPED? 


Yes - 

No  -  _ 

-14 - 

1 

- 9% 

- 

(Other/No  Response) 

—  2  - 

-  lo/o 

TRANSPONDER  EQUIPPED? 

Yes - 

145 - 

- 97% 

No - 

-  3 - 

- 2% 

(Other/No  Response) 

-  2 - 

-  lo/o 

HOURS  OF  CRUISE  CAPABLE 

3  Hours  - 

-16 - 

- 11% 

4  Hours - 

.  55 - 

- 37% 

5  Hours - 

.  48 - 

- 32% 

6  Hours  - 

-  15 - 

- lOo/o 

7  Hours - 

-  8  - 

- 50/0 

(Other/No  Response)  • 

-  8  - 

-  50/0 

DO  YOU  OWN  THIS  AIRCRAFT? 

Yes - 54  36% 

No - 94  63% 

(Other/No  Response)  —  2  -  1  % 
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Table  B-8.  Attribute  Experience  -  Terrain. 


FLOWN 

I  have 
I  have 
I  have 
I  have 
I  have 

FLOWN 
I  have 
I  have 
I  have 
I  have 
I  have 

FLOWN 
I  have 
I  have 
I  have 
I  have 
I  have 


OVER  LARGE  BODY  OF  WATER 

NEVER  flown  into  these  or  similar  conditions - 

flown  into  these  conditions  ONCE  or  TWICE  - 

flown  into  these  conditions  SEVERAL  times  (3-5) - 

flown  into  these  conditions  MANY  times  (6-10) - 

flown  into  these  conditions  NUMEROLfS  times  (11+) 

OVER  NON-MOUNTAINOUS  TERRAIN 

NEVER  flown  into  these  or  similar  conditions - 

flown  into  these  conditions  ONCE  or  TWICE  - 

flown  into  these  conditions  SEVERAL  times  (3-5) - 

flown  into  these  conditions  MANY  times  (6-10) - 

flown  into  these  conditions  NUMEROUS  times  (11+) 

OVER  MOUNTAINOUS  TERRAIN 

NEVER  flown  into  these  or  similar  conditions - 

flown  into  these  conditions  ONCE  or  TWICE  - 

flown  into  these  conditions  SEVERAL  times  (3-5) - 

flown  into  these  conditions  MANY  times  (6-10) - 

flown  into  these  conditions  NUMEROUS  times  (11+) 


Count 

Percent 

—  88 - 

- 59% 

—  19 - 

- 13% 

—  15 - 

- 10% 

—  1 1 - 

- 7% 

..  17 - 

- 11% 

—  7 - 

- 5% 

- 3 - 

- 2% 

—  1 1 - 

- 70/0 

...  1  1  - 

- 7% 

..118 - 

- 79% 

...  67 - 

- 45% 

..  21 - 

- 14% 

...  22 - 

- 150/0 

...  1  1  - 

- 70/0 

...  29 - 

- 190/0 

Table  B-9.  Attribute  experience  -  Precipitation 


LIGHT  RAIN 

Count  Percent 

I  have  NEVER  flown  into  these  or  similar  conditions - 3 - 2% 

I  have  flown  into  these  conditions  ONCE  or  TWICE  -  39 - 26% 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) -  42 - 28% 

I  have  flown  into  these  conditions  MANY  times  (6-10) -  25 - 17% 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) - 41 - 27% 

MODERATE  RAIN 

I  have  NEVER  flown  into  these  or  similar  conditions - 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) 

HEAVY  SNOW 

I  have  NEVER  flown  into  these  or  similar  conditions - 122 - 81% 

1  have  flown  into  these  conditions  ONCE  or  TWICE  -  1 1 - 7% 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 6 - 4% 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 5 - 3% 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) - 6 - 4% 


-  34 - 23% 

-  51 - 340^ 

—  31 - 21% 

- 8 - 5% 

- 26 - 17% 
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Table  B-10.  Attribute  Experience  -  Visibility. 

ONE  NAUTICAL  MILE 

I  have  NEVER  flown  into  these  or  similar  conditions - 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) 

FOUR  NAUTICAL  MILES 

I  have  NEVER  flown  into  these  or  similar  conditions  — - 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) 

EIGHT  NAUTICAL  MILES 

I  have  NEVER  flown  into  these  or  similar  conditions - 4 - 3% 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 4 - 3% 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) -  1 1 - 7% 

I  have  flown  into  these  conditions  MANY  times  (6-10) -  17 - 11% 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) -  114 - 76% 


12 - 8% 

19 - 130/0 

24 - 16% 

38- . 25% 

57 - 38% 


Count 

Percent 

—  42 - 

- 28% 

—  36 - 

- 24% 

...  32 - 

- 21% 

—  11 - 

- 70/0 

...  29 - 

- 19% 

Table  B-1 1 .  Attribute  experience  -  Ceiling 


800  FEET 

I  have  NEVER  flown  into  these  or  similar  conditions - 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) 

1800  FEET 

I  have  NEVER  flown  into  these  or  similar  conditions - 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) 

4000  FEET 

I  have  NEVER  flown  into  these  or  similar  conditions - 

I  have  flown  into  these  conditions  ONCE  or  TWICE  - 

I  have  flown  into  these  conditions  SEVERAL  times  (3-5) - 

I  have  flown  into  these  conditions  MANY  times  (6-10) - 

I  have  flown  into  these  conditions  NUMEROUS  times  (11+) 


Count 

Percent 

..45 — 

- 30% 

38 - 

- 25% 

..  20 - 

- 13% 

..  1  1 - 

- 7% 

-  36 - 

- 24% 

...  8 - 

- 5% 

.  17. - 

- 1 1  % 

.  27 - 

- 18% 

32 - 

- 21% 

66- - 

- 44% 

...3 - 

- 2% 

..  3 - 

- 2% 

..  8 - 

- 5% 

..  20 - 

- 13% 

116 - 

- 77% 
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APPENDIX  C 


EXPERT  PILOT  SAFETY  RATINGS 


Cl 


EXPERT  PILOT  SAFETY  RATINGS 


You  have  been  selected  as  a  Subject  Matter  Expert  (SME)  in  the  field  of  aviation 
safety  due  to  your  personal  flying  experience  and/or  your  FAA  job  experience. 

Place  yourself  in  this  scenario:  Kelly  (a  new  pilot  VFR-only  qualified  with  around 
500  flying  hours)  asks  you,  “How  safe  would  it  be  for  me  to  fly  this  route  VFR?” 

Use  this  scale  from  0  to  1 00  to  indicate  your  expert  safety  assessment  where: 

0  =  Absolutely  unsafe  for  Kelly  to  fly  VFR. 

1 00  =  Totally  safe  for  Kelly  to  fly  VFR. 

Note  1 :  Use  your  total  experience  (VFR  and  IMG)  to  evaluate  the  conditions  on 
the  following  pages.  NONE  of  the  choices  shown  may  deserve  a  “0”  for  “Absolutely 
unsafe”  or  a  “100”  for  “Totally  safe.” 

Use  any  numbers  in  the  range  from  0  to  100  to  indicate  your  expert  opinion,  but 
you  MAY  NOT  use  the  same  number  more  than  once  on  one  page.  If  you  feel  that  two 
choices  are  “Absolutely  unsafe,”  rate  one  as  “0”  and  the  other  as  “1”. 

You  and  we  know  that  evaluating  conditions  for  determining  the  safety  of  any 
flight  involves  interaction  between  all  weather,  aircraft,  pilot  and  terrain  variables.  Yet 
each  of  us  has  developed  benchmarks  for  certain  values  of  each  variable  on  which  we 
make  flight  planning  decisions.  We  ask  you  to  consider  each  variable  (i.e.,  TERRAIN, 
PRECIPITATION,  VISIBILITY  and  CEILING)  separately  and  give  us  your  opinion  on  the 
relative  safety  of  each  value  for  VFR  flying.  Treat  each  page  as  a  separate  and 
unrelated  rating  assignment. 

Read  each  of  the  following  pages  in  turn,  and  give  us  your  opinion  on  the  safety 
of  each  value  using  the  scale  at  the  top  of  the  page  and  using  each  number  only  once 
on  each  page. 

Note  2:  Values  are  not  necessarily  presented  in  order  of  “worst-to-best”  or  “best- 
to-worst.”  The  order  is  simply  to  help  us  ensure  that  all  target  values  are  covered. 
Remember,  you  are  evaluating  each  of  the  variables  against  the  safety  for  a  VFR 
qualified  pilot  with  around  500  hours  of  flying  time  to  fly  in  a  reliable  single  engine  VFR 
only  aircraft. 


Turn  the  page  and  begin. 
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TERRAIN 


Rate  each  route  on  how  safe  you  think  it  would  be  for  a  VFR  only  pilot 
with  around  500  flying  hours  experience  to  encounter.  Use  the  following  scale 
and  each  number  no  more  than  once: 

0  =  Absolutely  unsafe. 

100  =  Totally  safe. 

ID  Rating  Destination 

a.  _  Route  “1”  (City  County  to  Delta  County,  Ml) 

b.  _  Route  “2”  (Hereford  to  Dalhart,  TX) 

c.  _  Route  “3”  (Las  Vegas  to  San  Juan  Pueblo,  NM) 
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PRECIPITATION 


Rate  each  item  on  how  safe  you  think  it  would  be  for  a  VFR  only  pilot  with 
around  500  flying  hours  experience  to  encounter.  Use  the  following  scale  and 
each  number  no  more  than  once: 


0  =  Absolutely  unsafe. 


100 

=  Totally  safe. 

ID 

Rating  Description 

a. 

No  Precipitation 

b. 

Liaht  Rain 

c. 

Moderate  Rain 

d. 

Heavy  Rain 

e. 

Liqht  Snow 

f. 

Moderate  Snow 

g- 

Heavy  Snow 

h. 

Freezino  Rain 

VISIBILITY 


Rate  each  item  on  how  safe  you  think  it  would  be  for  a  VFR  only  pilot  with 
around  500  flying  hours  experience  to  encounter.  Use  the  following  scale  and 
each  number  no  more  than  once: 


0  =  Absolutely  unsafe. 

100  =  Totally  safe. 

ID  Rating  Description 


a. 

b. 

c. 

d. 

e. 

f. 

g- 

h. 


j- 

k. 


1/2  NM 
1  NM 

1  1/2  NM 
2NM 

2  1/2  NM 
3NM 
4NM 

5  NM 

6  NM 

7  NM 

8  NM 

More  than  8  NM 
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CEILING 


Rate  each  item  on  how  safe  you  think  it  would  be  for  a  VFR  only  pilot  with 
around  500  flying  hours  experience  to  encounter.  Use  the  following  scale  and 
each  number  no  more  than  once: 


0  =  Absolutely  unsafe. 


100 

=  Totally  safe. 

ID 

Rating  Description 

a. 

600  ft. 

b. 

700  ft. 

c. 

800  ft. 

d. 

900  ft. 

e. 

1000  ft. 

f. 

1200  ft. 

g- 

1400  ft. 

h. 

1600  ft. 

i. 

1800  ft. 

j- 

2000  ft. 

k. 

2500  ft. 

I. 

3000  ft. 

m. 

3500  ft. 

n. 

4000  ft. 

0. 

5000  ft. 

P- 

over  5000  ft 
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Personal  Experience  Profile 


Thank  you  for  your  expert  ratings.  Your  data  will  be  combined  with  data 
from  other  experts  to  form  the  foundation  for  studying  safety  judgment  of 
General  Aviation  (GA)  pilots. 

Now,  please  fill  out  the  PILOT  INFORMATION  and  four  EXPERIENCE 
Forms  which  ask  for  your  personal  flying  history.  These  forms  are  intended  only 
for  use  in  this  study  of  GA  pilots.  Notice  neither  your  name  nor  any  personal 
identifying  information  is  requested. 

Please  feel  free  to  write  suggestions  or  comments  on  these  sheets 
regarding  problems  we  may  expect  within  the  GA  community  with  these  forms. 

The  PILOT  INFORMATION  sheet  requests  personal  flying  history 
information  which  may  be  important  in  explaining  the  decision  making  process 
used  by  different  subgroups  within  the  GA  pilot  community. 

The  ATTRIBUTE  EXPERIENCE  forms  appear  similar  to  the  four  rating 
sheets  you  just  completed.  Instead  of  rating  “HOW  SAFE?”  you  are  asked  to 
indicate  how  much  experience  you  have  had  under  specific  conditions.  Because 
these  forms  are  designed  for  the  GA  pilot  community,  you,  as  a  senior  expert, 
may  well  be  responding  level  “5”  to  all  these  items  -  and  that  is  OK. 

Here  is  the  scale  used  on  all  four  Attribute  Experience  forms: 

0  =  I  have  NEVER  flown  into  these  or  highly  similar  conditions. 

1  =  I  have  flow  into  these  conditions  ONCE  or  TWICE 

2  =  (same  as  above) 

3  =  I  have  flown  into  these  conditions  SEVERAL  times. 

4  =  (same  as  above) 

5  =  I  have  flown  into  these  conditions  MANY  times. 

COMMENTS  and/or  SUGGESTIONS 
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EXPERIENCE:  TERRAIN 


How  much  personal  expeience  (VFR  and  IMC)  do  you  have  with  these 
conditions: 


0  =  I  have  NEVER  flown  into  these  or  highly  similar  conditions. 

1  =  I  have  flow  into  these  conditions  ONCE  or  TWICE 

2  =  (same  as  above) 

3  =  I  have  flown  into  these  conditions  SEVERAL  times. 

4  =  (same  as  above) 

5  =  I  have  flown  into  these  conditions  MANY  times. 


ID  Exper.  Destination 

a-  _  Route  “1”  (City  County  to  Delta  County,  Ml) 


b.  _  Route  “2”  (Hereford  to  Dalhart,  TX) 


c. 


Route  “3”  (Las  Vegas  to  San  Juan  Pueblo,  NM) 
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EXPERIENCE:  PRECIPITATION 


How  much  personal  expeience  (VFR  and  IMC)  do  you  have  with  these 
conditions: 


0  =  I  have  NEVER  flown  into  these  or  highly  similar  conditions. 

1  =  I  have  flow  into  these  conditions  ONCE  or  TWICE 

2  =  (same  as  above) 

3  =  I  have  flown  into  these  conditions  SEVERAL  times. 

4  =  (same  as  above) 

5  =  I  have  flown  into  these  conditions  MANY  times. 


ID  Exper. 

Description 

a. 

No  Precipitation 

b. 

« 

Light  Rain 

c. 

Moderate  Rain 

d. 

Heavy  Rain 

e. 

Light  Snow 

f. 

Moderate  Snow 

g- 

Heavy  Snow 

h. 

Freezing  Rain 
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EXPERIENCE:  VISIBILILTY 


How  much  personal  expeience  (VFR  and  IMC)  do  you  have  with  these 
conditions: 


0 

1 

2 

3 

4 

5 


I  have  NEVER  flown  into  these  or  highly  similar  conditions. 
I  have  flow  into  these  conditions  ONCE  or  TWICE 
(same  as  above) 

I  have  flown  into  these  conditions  SEVERAL  times. 

(same  as  above) 

I  have  flown  into  these  conditions  MANY  times. 


ID 

Exper. 

Description 

a. 

1/2  NM 

b. 

1  NM 

c. 

1  1/2  NM 

d. 

2  NM 

e. 

2  1/2  NM 

f. 

3NM 

g- 

4NM 

h. 

5NM 

I. 

6NM 

j- 

7NM 

k. 

8NM 

I. 


More  than  8  NM 


EXPERIENCE:  CEILING 


How  much  personal  expeience  (VFR  and  IMC)  do  you  have  with  these 
conditions: 

0  =  I  have  NEVER  flown  into  these  or  highly  similar  conditions. 

1  =  I  have  flow  into  these  conditions  ONCE  or  TWICE 

2  =  (same  as  above) 

3  =  I  have  flown  into  these  conditions  SEVERAL  times. 

4  =  (same  as  above) 

5  =  I  have  flown  into  these  conditions  MANY  times. 


ID 

Exper. 

Description 

a. 

600  ft. 

b. 

700  ft. 

c. 

800  ft. 

d. 

900  ft. 

e. 

1000  ft. 

f. 

1200  ft. 

g- 

1400  ft. 

h. 

1600  ft. 

i. 

1800  ft. 

j- 

2000  ft. 

k. 

2500  ft. 

I. 

3000  ft. 

m. 

3500  ft. 

n. 

4000  ft. 

0. 

5000  ft. 

P- 

over  5000  ft 

APPENDIX  D 


PLOTS  OF  EXPERT  PILOT  SAFETY  RATINGS 
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Safety  Ratings  of  Precipitation  by  Expert  Pilots 

None 
Light  Rain 
Light  Snow 
Moderate  Rain 
Moderate  Snow 
Heavy  Rain 
Heavy  Snow 
Freezing  Rain 

3.5  4  4.5  5  5.5  6  6.5 

Safety  Rating 


Safety  Ratings  of  Terrain  by  Expert  Pilots 


APPENDIX  E 


SELECTED  PAIR  COMPARISONS  AMONG 
POLICY  GROUPS  FOR  EACH  SCENARIO 

(All  mean  differences  statistically  significant  for  p  <  .05) 


El 


Highlights  of  Characteristics  for  Mountainous  Terrain  Groups: 


GPl 

S-N: 

S-N 

J-N 

51 

41 

36% 

59% 

S-N 

J-B 

51 

34 

2.43 

1.19 

3.50 

2.52 

36% 

68% 

2.00 

1.19 

43% 

16% 

1969 

561 

S-N 

S-C 

.83 

.60 

47 

21 

3.50 

2.50 

36 

17 

3.14 

2.25 

3.86 

3.28 

66% 

86% 

36% 

61% 

2.43 

1.58 

GP2 

J-N: 

J-N 

S-N 

41 

51 

59% 

36% 

J-N 

J-B 

29 

46 

41 

34 

39% 

16% 

3.10 

2.52 

10% 

6% 

1963 

561 

1.81 

1.19 

1.84 

1.19 

2.59 

2.10 

1.90 

1.35 

33% 

16% 

.80 

.75 

Senior-Nominal  Policy  Group:  (Mountainous) 

Compared  to  the  Junior-Nominal  group,  this  Senior-Nominal  group  has: 
Higher  average  Age 

Lower  percentage  report  PP  as  highest  certification 

Compared  to  the  Junior-Bold  group,  this  Senior-Nominal  group  has: 
Higher  average  Age 

Higher  experience  with  Ceiling  800  FT 
Higher  experience  with  Ceiling  1800  FT 
Lower  percentage  report  PP  as  highest  certification 
Higher  experience  with  Precip  Moderate  Rain 
Higher  percentage  own  aircraft 
Higher  average  Total  Hours  Flown 

Compared  to  the  Senior-Cautious  group,  this  Senior-Nominal  group  has: 
Higher  average  R-Squared  (Use  of  Expert  Info) 

Higher  average  Hours  Flown  Last  90  days 
Higher  experience  with  Ceiling  1800  FT 
Higher  averse  Comfort  Level 
Higher  experience  with  Visibility  4  NM 
Higher  experience  with  Ceiling  4000  FT 
Lower  percentage  fly  for  Pleasure 
Lower  percentage  report  PP  as  highest  certification 
Higher  experience  with  Ceiling  800  FT 

Junior-Nominal  Policy  Group:  (Mountainous) 

Compared  to  the  Senior-Nominal  group,  this  Junior-Nominal  group  has: 
Lower  average  Age 

Higher  percentage  report  PP  as  highest  certification 

Compared  to  the  Junior-Bold  group,  this  Junior-Nominal  group  has: 

Lower  average  Comfort  Level 

Higher  average  Age 

Higher  percentage  own  aircraft 

Higher  experience  with  Ceiling  1800  FT 

Higher  percentage  Female 

Higher  average  Total  Hours  Flown 

Higher  experience  with  Precip  Moderate  Rain 

Higher  experience  with  Ceiling  800  FT 

Higher  experience  with  Precip  Light  Rain 

Higher  experience  with  Visibility  1  NM 

Higher  percentage  report  COM/ATP  as  highest  cert. 

Higher  average  R-Squared  (Use  of  Expert  Info) 


J-N 

S-C 

Compared  to  the  Senior-Cautious  group,  this  Junior-Nominal  group  has: 

29 

17 

Higher  average  Comfort  Level 

.80 

.60 

Higher  average  R-Squared  (Use  of  Expert  Info) 

62% 

86% 

Lower  percentage  flying  for  Pleasure 

3.72 

3.28 

Higher  experience  with  Visibility  8  NM 

3.77 

3.28 

Higher  experience  with  Ceiling  4000  FT 

22% 

5% 

Higher  percentage  Primary  Job  as  Pilot 

49 

21 

Higher  average  Hours  Flown  Last  90  days 

2.94 

2.25 

Higher  experience  with  Visibility  4  NM 

3.10 

2.50 

Higher  experience  with  Ceiling  1800  FT 

41 

48 

Lower  average  Age 

1.90 

1.39 

Higher  experience  with  Visibility  1  NM 

GP3 

J-B: 

Junior-Bold  Policy  Group:  (Mountainous) 

J-B 

S-N 

Compared  to  the  Senior-Nominal  group,  this  Junior-Bold  group  has: 

34 

47 

Lower  average  Age 

1.19 

2.43 

Lower  experience  with  Ceiling  800  FT 

2.52 

3.50 

Lower  experience  with  Ceiling  1800  FT 

16% 

36% 

Lower  percentage  report  COM/ATP  as  highest  cert. 

68% 

36% 

Higher  percentage  report  PP  as  highest  certification 

1.19 

2.00 

Lower  experience  with  Precip  Moderate  Rain 

16% 

43% 

Lower  percentage  Own  Aircraft 

561 

1969 

Lower  average  Total  Flying  Hours 

J-B 

J-N 

Compared  to  the  Junior-Nominal  group,  this  Junior-Bold  group  has: 

46 

29 

Higher  average  Comfort  Level 

16% 

39% 

Lower  percentage  Own  Aircraft 

34 

41 

Lower  average  Age 

2.52 

3.10 

Lower  experience  with  Ceiling  1800  FT 

561 

1963 

Lower  average  Total  Flying  Hours 

1.19 

1.81 

Lower  experience  with  Precip  Moderate  Rain 

1.19 

1.84 

Lower  experience  with  Ceiling  800  FT 

2.10 

2.59 

Lower  experience  with  Precip  Light  Rain 

1.35 

1.90 

Lower  experience  with  Visibility  1  NM 

16% 

33% 

Lower  percentage  report  COM/ATP  as  highest  cert. 

.75 

.80 

Lower  average  R-Squared  (Use  of  Expert  Info) 

J-B 

S-C 

Compared  to  the  Senior-Cautious  group,  this  Junior-Bold  group  has: 

46 

17 

Higher  average  Comfort  Level 

34 

48 

Lower  average  Age 

16% 

44% 

Lower  percentage  Own  Aircraft 

.75 

.60 

Higher  average  R-Squared  (Use  of  Expert  Info) 

6% 

25% 

Lower  percentage  Female 

5% 

17% 

Lower  percentage  use  Weather  Source:  Other 

16% 

33% 

Lower  percentage  report  COM/ATP  as  highest  cert. 

561 

2140 

Lower  average  Total  Flying  Hours 

E4 


GP4 

S-C 

.60 

21 

2.50 

17 

2.25 

3.28 

86% 

61% 

1.58 

S-C 

17 

.60 

86% 

3.28 

3.28 

5% 

21 

2.25 

2.50 

25% 

48 

1.39 

S-C 

17 

48 

44% 

.60 

25% 

17% 

33% 

2140 

Highlights 

GP-A 

T-N 

27 

1% 

1.71 

GP-B 

T-B 

40 

5% 

1.29 


S-C:  Senior-Cautious  Policy  Group:  (Mountainous) 

S-N  Compared  to  the  Senior-Nominal  group,  this  Senior-Cautious  group  has: 

.83  Lower  average  R-Squared  (Use  of  Expert  Info) 

47  Lower  average  Hours  Flown  Last  90  days 

3.50  Lower  experience  with  Ceiling  1800  FT 

36  Lower  average  Comfort  Level 

3.14  Lower  experience  with  Visibility  4  NM 

3.86  Lower  experience  with  Ceiling  4000  FT 

66%  Higher  percentage  flying  for  Pleasure 

36%  Higher  percentage  report  PP  as  highest  certification 

2.43  Lower  experience  with  Ceiling  800  FT 

J-N  Compared  to  the  Junior-Nominal  group,  this  Senior-Cautious  group  has: 

29  Lower  average  Comfort  Level 

•  8  0  Lower  average  R-Squared  (Use  of  Expert  Info) 

62%  Higher  percentage  flying  for  Pleasure 

3.72  Lower  experience  with  Visibility  8  NM 

3.77  Lower  experience  with  Ceiling  4000  FT 

22%  Lower  percentage  Primary  Job  as  Pilot 

49  Lower  average  Hours  Flown  Last  90  days 

2.94  Lower  experience  with  Visibility  4  NM 

3.10  Lower  experience  with  Ceiling  1800  FT 

10%  Higher  percentage  Female 

4 1  Higher  average  Age 

1.90  Lower  experience  with  Visibility  1  NM 

J-B  Compared  to  the  Junior-Bold  group,  this  Senior-Cautious  group  has: 

46  Lower  average  Comfort  Level 

34  Higher  average  Age 

16%  Higher  percentage  Own  Aircraft 

.75  Lower  average  R-Squared  (Use  of  Expert  Info) 

6%  Higher  percentage  Female 

5%  Higher  percentage  use  Weather  Source:  Other 

16%  Higher  percentage  report  COM/ATP  as  highest  cert. 

561  Higher  average  Total  Flying  Hours 

of  Characteristics  for  Over-Water  Terrain  Groups: 

T-N:  Total-Nominal  Policy  Group:  (Over  Water) 

T-B  Compared  to  the  T-B  group,  this  T-N  group  has: 

40  Lower  average  Comfort  Level 

5%  Lower  percentage  use  Weather  Source:  None 

1.29  Higher  experience  with  Precip  Moderate  Rain 

T-B:  Total-Bold  Policy  Group:  (Over  Water) 

T-N  Compared  to  the  T-N  group,  this  T-B  group  has: 

27  Higher  average  Comfort  Level 

1  %  Higher  percentage  use  Weather  Source:  None 

1.71  Lower  experience  with  Precip  Moderate  Rain 
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COMPREHENSIVE  OCCUPATIONAL 
DATA  ANALYSIS  PROGRAM  (CODAP) 
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Comprehensive  Occupational  Data  Analysis  Program  (CODAP) 

CODAP  is  a  suite  of  software  products  developed  by  the  US  Air  Force  for  the  analysis  of  large  bodies 
of  occupational  data,  commonly  obtained  from  job  analyses  and  other  studies  of  elements  of  the  Air  Force 
personnel  system.  In  addition  to  its  use  by  the  Air  Force,  CODAP  has  also  been  adopted  for  use  by  foreign 
and  domestic  governmental  units,  including  the  Australian  government  and  the  state  of  Maryland.  One 
use  of  CODAP  which  is  particularly  relevant  to  the  current  study  is  its  application  in  studies  of  promotion 
boards.  The  linear  modeling  capabilities  of  CODAP  components  allow  for  the  identification  of  the 
worth  functions  used  by  promotion  boards  in  the  evaluation  of  individuals  for  promotion.  This  process, 
which  is  often  termed  “policy-capturing”  allows  for  the  explication  of  the  relative  weighting  which  the 
board  and  the  constituent  members  have  applied  to  the  available  demographic,  training,  performance, 
and  other  data  which  are  provided  for  each  individual  eligible  for  promotion  in  arriving  at  their  global 
promote  versus  do-not-promote  decision.  In  some  ways  this  is  highly  similar  to  the  decision  process 
undertaken  by  pilots  in  evaluating  the  many  elements  of  weather  information  prior  to  making  their  go 
versus  no-go  decision.  A  similar  technique  (conjoint  analysis)  has  been  used  in  previous  studies  of  aviator 
decision  making  (Flathers,  Giffin,  and  Rockwell;  1982). 

GRPREL:  (pronounced  “Group”  “Rel”)  A  standard  interrater  reliability  program  in  the  CODAP 
system.  For  a  given  list  of  items  rated  by  a  set  of  Subject  Matter  Experts  (SMEs),  this  program  reports 
two  measures  of  interrater  agreement  (R^j  and  Ry^).  The  R^  value  indicates  the  reliabilitly  ofthe  observed 
set  of  ratings  —  while  0 . 1 0  is  considered  a  minimum  for  usable  rater  agreement,  a  value  of  0.20  or  greater 
is  desired.  The  R^^  value  is  driven  by  the  number  of  raters  acutally  used.  Although  an  R^|^of0.90  is  usually 
desired,  it  may  not  be  practical  in  a  particular  study  because  of  a  small  number  of  raters  (SMEs)  that  may 
be  subdivided  even  further  into  smaller  groups  based  on  policy  differences.  The  GRPREL  program  also 
computes  means  and  standard  deviations  for  each  item  in  the  list.  Item-level  reports  are  printed  in  three 
orders:  original  sequence,  ordered  descending  y  mean  value,  and  ordered  descending  on  standard 
deviation.  GRPREL  computes  each  rater’s  correlation  with  the  full-group  mean  vector  and  uses  a 
probability  evaluation  to  recommend  the  removal  of  deviant  (non-cooperative  or  reversed  scale)  raters. 
The  program  can  automatically  iterate  and  remove  flagged  raters  until  either  a  sufficient  level  of 
agreement  (Rj^j^  =  0.90)  is  reached  or  no  raters  can  be  found  with  a  probability  (of  deviant  rating)  above 

0.95. 

An  overview  of  the  CODAP  system  is  provided  by  Christal  ( 1 974),  and  a  complete  description  of  the 
interrater  reliability  components  is  given  in  Staley  and  Weissmuller  (1981).  The  interested  reader  is 
directed  to  those  sources  for  a  more  complete  technical  description  of  these  software  analytic  tools. 
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